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SUMMARY 


Although the aircraft checklist has long been regarded as the foundation of pilot standardization and 
cockpit safety, it has escaped the scrutiny of the human factors profession. The improper use, or 
the non-use, of the normal checklist by flight crews is often cited as the probable cause or at least a 
contributing factor to aircraft accidents. 

In this report the authors attempt to analyze the normal checklist, its functions, format, design, 
length, usage, and the limitations of the humans who must interact with it. The development of the 
checklist from the certification of a new aircraft to its delivery and use by the customer is 
discussed. The influence of the government, particularly the FAA Principal Operations Inspector 
(POI), the manufacturer's philosophy, the airline's "culture," and the end user- the pilot, all 
influence the ultimate design and usage of this device. The effects of airline mergers and 
acquisitions on checklist usage and design are noted. In addition, the interaction between 
production pressures ("making schedules"), checklist usage and checklist management are 
addressed. Finally, the authors provide a list of design guidelines for normal checklists. 
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1. INTRODUCTION 


FAR 121.315 Cockvit Check P rocedure 

(a) Each certificate holder shall provide an approved cockpit check procedure for each type 
of aircraft. 

(b) The approved procedures must include each item necessary for flight crew-members to 
check for safety before starting engines, taking off, or landing, and in engine and system 
emergencies. The procedure must be designed so that a flight crewmember will not need to 
rely upon his memory for items to be checked. 

(c) The approved procedures must be readily usable in the cockpit of each aircraft and the 
flight crew shall follow them when operating the aircraft. 


1.1. THE NORMAL CHECKLIST 

Historically, the first checklist was probably implemented as one conscientious pilot stepped out of 
his plane with some remarks about complexity and forgetfulness, realizing that he could no longer 
commit to memory all the required steps of configuring a complicated aircraft. Yet, aviation has 
advanced from those early days, when a statement such as "do not trust any altitude instrument” 
was included on checklists of U.S. Air Service airplanes (Air Service, 1920), to an era when on- 
board computers calculate and execute precise vertical navigation maneuvers. However, checklists 
have not undergone any conceptual rethinking or design changes during this period, and in this 
respect, a B-32 checklist from 1943 (Figure 1) is very similar in its concept and design to a modem 
airliner checklist. 

The normal flight-deck checklist depicts a set of different tasks the pilot must perform or verify in 
order to configure the aircraft and prepare the crew for certain macro-tasks such as ENGINE 
START, TAXI, TAKEOFF, etc. For each one of these macro-tasks there are several "items” to be 
accomplished and verified by the flight crew. Although several methods of conducting a checklist 
are used by the airlines, most checklist formats follow the same strategy: 

1 . Reading or hearing the checklist item. 

2 . Accomplishing the item-either by verification of the correct setting or by execution of the 
checklist item. 

3 . Responding to the outcome of the action performed. 

While conducting this research, the authors encountered several occasions when the following 
statement was made, "checklists, they are simple and straightforward, so what is there to study 
about them?" However, a closer look into the usage of checklists and the controversy that 
surrounds them will reveal a device and an associated procedure that, in addition to its basic 
function as a memory guide, is a generator and coordinator of many cockpit tasks. Nevertheless, 
its importance and vulnerability have long been neglected. 

In the space of 15 months, there have been three accidents where the misuse of a checklist was 
determined as one of the probable causes of accident by the National Transportation Safety Board 
(NTSB). In the first accident, an Air New Orleans BAe J-31 commuter airplane crashed 
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B-32 CHECK LIST 

Suitable For Use With 100 Octane Fuel Only 


BEFORE ENTERING AIRPLANE 

Visual Inspection of Airplane 
Pitot Head Cover Removed 
Tire and Oleo Inflation 
Wheel Chocks in Place 
Trim Tabs Neutral 
Crew Inspection 

BEFORE STARTING ENGINES 

Landing Gear Switch — NEUTRAL 
Forms I and IA 
Fuel and Oil 

Loading, WITHIN C.G. LIMITS 
Ignition OFF 

Props PULL THROUGH 6 BLADES 
Control Movement FREE 
Altimeter — SET 
Battery Switches — ON 
A.P.U. — Start, Equalizer Switch OFF 
Inverter Switch — MAIN ON 
Prop. Feather Switches — NORMAL 
Prop. Reverse Safety Switches — SAFE 
Prop. Reverse Pitch Switch — 
NORMAL 

Prop. Selector Switches — AUTO- 
MATIC 

Prop. Speed Control — 2800 
Prop. Master Motor Switch — ON 
All Circuit Breakers — ON 
Throttle — 1000 R.P.M. Position 
Turbo Boost Selector — 0 
Mixtures Controls — IDLE CUT-OFF 
Intercooler Flaps — AUTOMATIC 
Oil Cooler Flaps — AUTOMATIC 


Carburetor Air Filters — AS RE- 
QUIRED 

Carburetor Heat — OFF 
Anti-Icers, Wings and Props — OFF 
Electrical Hyd. Pump Switch — ON 
Parking Brakes — ON 
Hydraulic Brake Pressure — CHECK 

STARTING ENGINES 

Fuel Selector Valves — TANK TO 
ENGINE 

Booster Pumps — ON LOW 

(No Fuel Pressure Ind. until Mixture 
Control is Moved) 

Fire Guard and Call CLEAR 
Master Ignition Switch — ON 
Ignition Switch — ON AFTER TWO 
PROP. REVOLUTIONS 
Mixture— AUTO RICH AFTER 
ENGINE IS RUNNING 

WARM UP 

Fuel and Oil Pressures 
Booster Pumps — OFF 
Vacuum and Flight Indicator 
Generators — ON, 28V 
A.P.U. Equalizer Switch — ON 
Inverter — CHECK 
Wing Flaps — OPERATE 
Prop. Control— CHECK R.P.M. 
CHANGE 

Magneto — CHECK at 2000 R.P.M. 


Figure 1. Courtesy of the Smithsonian National Air and Space Museum Branch Library 
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immediately after takeoff, due to flight-crew failure to apply maximum takeoff power (NTSB, 
1988b). In the second accident. Northwest Airlines Flight 255, an MD-80 crashed shortly after 
takeoff from Detroit Metro Airport (Michigan) following a no-flap/no-slat takeoff (NTSB, 1988a). 
In the third accident, Delta Air Lines Flight 1 141, a B-727 crashed shortly after lifting off from 
runway 18L at Dallas-Fort Worth International Airport (Texas), after a no-flap/no-slat takeoff 
(NTSB, 1989). Further, a ongoing investigation of USAir Flight 5050, a B-737-400 which ran off 
LaGuardia Airport Runway 31 and dropped into adjacent waters, is also focusing on the crews’ 
checklist performance ( Aviation Week and Space Technology , 1990, April 2). 

In testimony given before the Safety Board investigating the Northwest Airlines Flight 255 
accident, the second author of this report testified with respect to checklist presentation "that he did 
not know of any human factors research on how a checklist should be designed..." (NTSB, 

1988a, p. 62). The same dearth of research pertaining to use and design of current checklists (as 
opposed to future checklist design), was encountered by the authors while performing an intensive 
database search of U.S. and Western European sources. 

The Safety Board recognized the importance of checklist use and its critical role in the safety of 
flight operations in a 1969 recommendation following a Pan American World Airways B-707 crash 
after a no-flap takeoff (NTSB, 1969). This recommendation called for "Air carrier cockpit 
checklists to be reviewed in an effort to ensure that each list provides a means of reminding the 
crew, immediately prior to takeoff, that all items critical for safe flight have been accomplished" 
(NTSB recommendation A-69-012). 

Unfortunately, it took 18 years and a major accident before the Safety Board recognized the 
problems with the human factors aspects of checklist display and procedures. Following the 
Northwest Flight 255 accident and the commuter accident, the NTSB recommended that the 
Federal Aviation Administration (FAA) convene a human-performance research group to determine 
"...if there is any type or method of presenting a checklist which produces better performance on 
part of user personnel"; and for the FAA to recommend checklist typography criteria for 
commercial operators (NTSB recommendations A-88-068 and A-88-072). 

Following, the FAA sent a circular to commercial operators requiring FAA inspectors to review 
checklists and cockpit procedures for compatibility with "...airplane manufacturer 
recommendations, type certificate (TC), or STC holders' current recommendations," and with the 
requirements of FAR 121.315 (Air Carrier Operation Bulletin No. 8-88-4). Another circular, 
requires that Federal Aviation Regulations (FAR) Part 135 checklists "...should include the 
appropriate actions necessary for normal and emergency procedures, printed in clear, concise, and 
legible form" (Air Carrier Operations Bulletin Part 135 No. 88-5). However, no information 
regarding what is "clear, concise and legible form" is detailed or explained in this bulletin. 

As a result, the authors began this study by concentrating on the human factors of a paper checklist 
as a display per se. However, the evolution of this research demonstrated that this is only the outer 
shell of the problem. The core of the problem emerged as the design concepts and the social issues 
surrounding the use of the checklist that have led some pilots to misuse it or not use it at all. 

Checklist problems have been known to prevail in several other high risk industries such as the 
marine and the process industries. The capsizing of the English ferry Herald of Free Enterprise is a 
typical example of a marine checklist accident. The vessel departed the loading ramp in Zeebrugge 
port (Belgium) on March 6, 1987, with the bow doors unintentionally left open. As the ship 
increased speed to 18 knots just outside the harbor, water entered and flooded the lower car deck, 
leading to rapid capsize. 150 passengers and 38 crew members lost their lives due to an omission 
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of a pre-departure check item (Department of Transport, 1987). In the nuclear industry, °P e ^ors 
also use paper checklists for normal and abnormal procediues as well as for scheduled i^ntemmce 
tasks, leading also to checklist errors and omissions (Swam and Guttman, 1983, H. F. Van cott, 
personal communication, 1988). 


While the checklist problem is common to several transportation and chemical process industries, it 
is traditional that those industries look to the aviation industry for guidance m such common 
problems. 


1.2. OBJECTIVES 

The objectives of this study are as follows: 

1 . To understand the role of the checklist in the operation of a modem air transport aircraft. 

2. To identify the factors that contribute to the misuse or non-use of checklists. 

3 . To present guidelines for checklist design. 

11 METHODS 

The intent of this research is to further understand the role of the flight-deck checklist within the 
socio-technical matrix of procedures, operators, cockpit systems, management, and regulating 
agency (the FAA). Information concerning this study was obtained from the following resources: 

1 . Field studies. 

2. Interviews with "line” pilots from seven major U.S. earners. 

3 . Incident/accidents reports obtained from the following agencies and organizations: 

a) NASA's Aviation Safety Reporting System (ASRS). 

b) National Transportation Safety Board (NTSB). 

c) International Civil Aviation Organization (ICAO). 

4. Interviews with officials from government agencies (FAA, NTSB). 

5 . Information obtained from aircraft and avionics manufacturing companies. 

6. General literature in the fields of aviation, psychology, typography, and human 
performance. 

1.3.1. Field Studies 

Field studies of checklist performance were conducted atone U.S. earner which uses short- and 
medium-range aircraft (B-737 and B-757). The field studies were aimed at observing (from the 
cockpit iumpseat) flight crews in their daily line operation. A total of 42 different crews were 
observed to obtain the data; this amounted to 72 flights (legs) and totaled about 140 flight hours. 
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In order not to bias the data, and in concurrence with the company's flight management officials, 
the crews were not told of the specific purpose of the observations. In addition, similar 
observations were conducted in flight simulators of the Airbus A-310 and A-320. 

1.3.2. Interviews with Line Pilots 

Face-to-face interviews with line pilots from seven major U.S. earners were conducted by the 
authors and coordinated by the Air Line Pilots Association (ALP A). The sample consisted of ten 
captains and five first officers. We asked the pilots to explain the method of conducting the 
checklist at their company, and then posed several questions regarding checklist procedures. Pilots 
were assured that this information would not be identifiable and that the interview notes would be 
destroyed upon completion of the project. During jumpseat observation, no record was kept of 
flight crew names, flight numbers, or dates, to ensure confidentiality. 

1.3.3. Incidents/Accidents Databases 

NTSB. Documented information regarding checklist-related accidents and incidents for 
U.S. carriers was obtained from the NTSB accident/incident database (1983-1988) (NTSB, 

1988c), and the NTSB recommendation database (1969-1988) (NTSB, 1988d). This information 
was supplemented by obtaining published Aviation Accident Reports (AAR) and field reports from 
the agency. 

ICAO A similar search was conducted on the ICAO database which contains world-wide 
accident reports (ICAO, 1988). This database consisted of 12,000 reports representing accidents to 
aircraft above 2250 kg. reported to the ICAO since 1970. 

Although on first glance it seemed beneficial to conduct a statistical analysis on the above reports, 
subsequently this was not done. The authors' position was that such analysis would not aid in 
determining the factors that contribute to the checklist incident, mainly because most reports detail 
the outcome of the accident/incident, and not what type of checklist error contributed to the 
accident. 

ASRS. Another unique source of information regarding field operations is NASA’s 
Aviation Safety Reporting System (ASRS). This organization and its database utilize a voluntary 
reporting system where pilots, controllers, and others can submit subjective accounts about safety- 
related aviation incidents. The information derived from this database, since reporting is voluntary , 
may reflect reporting biases. 

Nevertheless, the power of the ASRS lies in the report narrative. Here pilots detail incidents and 
situations; they explain what happened, why it happened, and sometimes add suggestions for 
improvements. This database is very useful for identifying significant problems and potential 
solutions for operational procedures, cockpit systems design, and certification (McLucas, 
Drinkwater, and Leaf, 1981). 

The information obtained from a search on the ASRS database (1981-1989) (ASRS, 1987; ASRS 
1989a), was not analyzed statistically for the reasons detailed above and because of the reporting 
biases concealed in the database. The narratives used in this paper are not representative of all 
checklist-related reports. Rather, the narratives quoted in this paper illustrate problems associated 
with the use of checklists in air transport operations. 
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2. THE NATURE OF CHECKLISTS 

In this section we shall discuss the major concepts of checklist use and their objectives. In 
addition, we shall review the various checklist devices currently used by the airlines and the 

military. 


9.1 WHAT IS A CHECKLIST? 

The major function of the checklist is to ensure that the crew will properly configure the plane for 
flight, and maintain this level of quality throughout the flight, and in every flight. The process ot 
conducting a checklist occurs during all flight segments and, in particular, pnor to the critical 
segments (TAKEOFF, APPROACH, LANDING). Although these segments comprise only 27 per 
cent of average flight duration, they account for 76.3 per cent of hull-loss accidents (Lautman and 
Gallimore, 1988). 


2.1.1 Checklist Objectives 

Generally the checklist is intended to achieve the following objectives: 

1 . Aid the pilot in recalling the process of configuring the plane. 

2. Provide a standard foundation for verifying aircraft configuration that will defeat any 
reduction in the flight crew's psychological and physical condition. 

3 Provide convenient sequences for motor movements and eye fixations along the cockpit 
panels. 

4 . Provide a sequential framework to meet internal and external cockpit operational 
requirements. 

5 . Allow mutual supervision (cross checking) among crew members. 

6. Enhance a team (crew) concept for configuring the plane by keeping all crew members in 
the loop." 

7 Dictate the duties of each crew member in order to facilitate optimum crew coordination as 
well as logical distribution of cockpit workload. 

8 . Serve as a quality control tool by flight management and government regulators over the 
pilots in the process of configuring the plane for the flight. 

Another objective of an effective checklist design, often overlooked, is the promotion of a positive 
"attitude" toward the use of this procedure. For this to happen, the checklist must be well grounded 
within the "present day" operational environment, and the operator must have a sound realization 
of its importance instead of regarding it as a nuisance task (Nagano, 1975). 

From the above objectives, the checklist can be viewed in human factors terms as an additional 
interface between the human and the machine. This interface controls the method and sequence of 
verifying the plane's configuration. This is why the normal flight-deck checklist transformed from 
a simple memory-aid to a task by itself, with its own inherent advantages and disadvantages. 
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2. 1 .2. Abnormal and Expanded Checklist 

In addition to the normal checklist, other checklists are also used on the flight-deck and during 
training. These are the abnormal checklists and the expanded checklists (the term abnormal is 
broadened here to include non-normal and emergency checklists). The abnormal checklist is 
intended to aid the pilot during emergencies and/or malfunctions of the aircraft systems. To cope 
with such situations, the abnormal checklist serves to: 

1 . Act as memory guide. 

2 . Ensure that all critical actions are taken. 

3 . Reduce variability between pilots. 

4. Enhance coordination during high workload and stressful conditions. 

From the similarity of the above objectives to the objectives of the normal checklist, it is clear that 
there is much common between the concept and design of these checklists. The principal 
difference, however, lies in frequency of use. The abnormal checklist is very rarely performed by 
flight-crews during revenue flight; pilots are aware of its criticality, and very much aware that 
misuse or non-use of the abnormal checklist can transform a routine abnormality into an accident. 
The same cannot always be said about the normal checklist. 

Both checklists are part of the Standard Operating Procedures (SOPs) of the aircraft, as operated 
by the airline. In most airlines, the flight checklist is presented in the cockpit as a simple paper 
card, while the emergency checklist is detailed in the SOP manual or a Quick Reference Handbook 
(QRH). Many carriers include elaborate explanations of the normal and the abnormal checklists in 
the flight operation manual or the training manual. This document, called the expanded checklist, 
follows the same steps as the normal and abnormal checklists, but in more detail. The expanded 
checklist is used for training and as a supplement to the normal and abnormal checklists. It is not 
meant to be referred to in flight. 


2 2 CHECKLIST DEVICES 

Various types of checklist devices have evolved over the years; they range from use of mnemonics 
to computer-aided checklists. We shall discuss the use of each type, and note their respective 
advantages and disadvantages. 

2.2.1. Paper Checklist 

This is the most common type of checklist used today in commercial operations. Because of the 
prevalence of this type, it will be the focus of this report 

The paper checklist is a very simple device; it consists of a list of items written on a paper card 
(Figure 2). In most cases, the card is held in the pilot’s hand, or clipped to the yoke. In other 
cases, it is glued to the instrument panel or written on a placard attached to the yoke. 

There are several disadvantages to the use of a paper checklist. The main one is the lack of a 
pointer to distinguish between accomplished and non-accomplished items. Other disadvantages are 
the lack of a memory system to store unaccomplished items, and the need to occupy one hand in 
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Northwest Airlines MD-80 checklist 


NORTHWEST 


MD-80 


EXTERNAL ELECTRIC & PNEUMATIC SOURCE - START 

PNEUMATIC X- FEEDS BOTH CLOSED 

PNEUMATIC AIR SOURCE CONNECTED & ON 

PNEUMATIC X -FEEDS OPEN 

PNEUMATIC PRESSURE (25 PSI MIN) CKD 

COMPLETE - BEFORE START CHECKLIST 

AFTER ENGINES STABILIZED 

PNEUMATIC X-FEEDS BOTH CLOSED 

ELECTRIC POWER *CKD 

EXTERNAL ELECTRIC & PNEUMATIC . . . DISCONNECTED 
COMPLETE - AFTER START CHECKLIST 


BEFORE START 

BRAKES SET 

WINDSHIELD HEAT *ON 

FUEL PUMPS *( AS REQ) 

CABIN PRESSURE CONTROLLER *SET 

AUX HYDRAULIC PUMP & PRESSURE *ON & CKD 

CIRCUIT BREAKERS *+CKD 

AUTOLAND CKD 

TAKEOFF WARNING CKD 

RADIOS, ALTIMETERS & FLIGHT DIR **CKD & SET 

FUEL & OIL **( QUANTITIES) & RESET 


IGNITION ON 

SEAT BELT SIGN ON 

BEACON ON 


AFTER START 

ANNUNCIATOR CKD 

IGNITION *OFF 

ELECTRIC POWER *CKD 

APU AIR *(AS REQ) 

AIR CONDITIONING SUPPLY SWITCHES *AUTO 

PNEUMATIC X-FEED *ONE CLOSED 


TRANSFER PUMP & HYDRAULIC SYSTEMS *ON & CKD 


TAXI 

FLAPS 

TRIM 

EPR & AIRSPEED BUGS 

ARTS 

FLIGHT INSTRUMENTS 

CONTROLS & ELEVATOR POWER 


**( SETTING) 

♦♦(SETTING) 

♦♦(SETTINGS) 

(AS REQ) 

** ( HDG ) & SLAVING 

*CKD-TOP 

CKD-BOTTOM 


DELAYED ENGINE START 

BRAKES & IGNITION (AS REQ) & ON 

DELAYED AFTER START 


ANNUNCIATOR CKD 

IGNITION *OFF 

ELECTRIC POWER *CKD 

APU AIR *OFF 


AIR CONDITIONING SUPPLY SWITCHES . . . *AUTO 


ENGINE ANTI -ICE & FUEL HEAT (AS REQ) 

PNEUMATIC X-FEEDS *CLOSED 

APU *( AS REQ) 


BEFORE TAKEOFF 


FLIGHT ATTENDANT *NOTIFIED 

TRANSPONDER/TCAS *(AS REQ) 

ANNUNCIATOR CKD 

IGNITION ON 


CLIMB 


NO SMOKE SIGN *(AS REQ) 

IGNITION *( AS REQ) 

FUEL PUMPS *(AS REQ) 

CABIN PRESSURE CONTROLLER *CKD 

SYNC *ON 

HYDRAULIC PUMPS *OFF & LOW 

FLAP TAKEOFF SELECTOR ♦STOWED 


IN-RANGE 

ALTIMETERS 

EPR 

AIRSPEED BUG 

SEAT BELT SIGN 

CABIN PRESSURE CONTROLLER . . 
HYDRAULIC PUMPS 


♦♦(SETTING) & X-CKD 

*(GA) 

*♦( SETTING) 

*ON 

*CKD 

*ON & HIGH 


Figure 2. From NTSB, 1988a, Appendix E, p.138 
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holding the checklist. Paper checklists can also be difficult to read at night in low levels of cockpit 
illumination (Degani, 1988). 


2.2.2. Scroll Checklist 

The scroll checklist consists of a narrow strip of paper that scrolls vertically between two reels. 
The reels and paper are contained inside a box fitted with a window and a lubber line. Alter 
completing an item on the checklist the pilot rotates the reels to position the next item on the lubber 
line (Figure 3). This type of checklist is common in United States Air Force (USAF) transport 
aircraft (C-9, C-130, C-141, C-5). 


The main advantage of the scroll checklist is that it has a pointer system. One disadvantage of the 
scroll checklist is that, due to its relatively small size and orientation, it is difficult for the pilot to 
see the checklist devices which are mounted on the copilot's and on the flight engineer s panels. 
Another disadvantage is the lack of a memory system for unaccomplished items. Neverthe css, 
these checklist devices are highly regarded by military pilots (G. Sexton, personal communication, 

October, 1988). 


2.2.3. Mechanical and Electromechan ical Checklists 

A mechanical checklist consists of a small panel that contains several plastic slides moving over a 
list of checklist items (Figure 4). As the item is accomplished, the slide is moved to cover the 
item's nomenclature. Consequently, only the non-accomplished items are displayed. 

Very similar in concept is the electromechanical checklist. This device is made of a small panel with 
an internally-lighted list of items. Alongside each checklist item, a toggle switch is mounted. When 
the item is accomplished, the switch is turned off, and the light below the item s nomenclature is 
extinguished to indicate that the item has been completed (Figure 5). 

The mechanical and electromechanical devices are used by only one major U.S. earner for the 
BEFORE TAKEOFF and LANDING task-checklists. However, the rest of the task-checklists are 
performed from a paper card. The advantage of these checklists is that they have a pointer system, 
and allow clear presentation of skipped items. 


2.2.4. Vocal Checklist 

A vocal checklist is a unit that generates audible checklist calls preprogrammed by the manufacturer 
or the user. Using a rotary switch, the pilot can choose between different normal and abnormal 
task-checklists (Figure 6). 

Two push buttons- "proceed" and "acknowledge" are mounted on the yoke. Once an item is 
completed and "acknowledged," the "proceed" button is pressed to allow the next item to be 
generated. If a "proceed" is pressed without prior acknowledgment, the device will repeat the 
checklist item once again. The pilot can also intentionally skip an item and save it for later recall. 
The saved item will be move to the bottom of the list and will be generated at a later time. 

One company that manufactures this unit has made provisions to prioritize the aircraft radio and 
cockpit communication (intercom) over the audible checklist. However, the major drawback of this 
system is that it's a totally audio system. The checklist audio can be masked and blended into 
cockpit communications (ATC, company, ground, etc.), and vice versa. 



Scroll checklist 



















Electromechanical checklist 



Figure 5. Courtesy of American Airlines 
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Vocal checklist 



Figure 6. Courtesy of Flight Technology international 


2.2.5. Computer-Aid ed Checklists 

With the introduction of alphanumeric and graphic displays to the airline cockpit, it has become 
possible to include the checklist on these displays. There are two distinct categories of computer- 
aided checklists. The first is merely a display and pointer system, while the second is a display and 
a pointer system that is part of the feedback loop . In other words, the computer senses the 
system's status and feeds this information back to the screen. 

Display and pointer checklist. In the majority of these systems the pilot can choose 
between the emergency mode and the normal mode. In each mode an index page with all 
task-checklists is presented. Using a cursor, the pilot selects desired checklist from the index page. 

After selection of the task-checklist, checklist items appear on the screen (Figure 7). As the cursor 
is moved to the item being considered, the color of the item changes. Once the item is executed by 
the pilot and acknowledged, the color of the item will change again to indicate a completed item. 
Intentionally skipped items remain in the initial color and can be recalled later. In most systems, the 
user cannot advance to the next task-checklist until all skipped items are recalled and checked. 
Nevertheless, in one company's device, it is possible to switch between different task-checklists 
before completing them. However, this action will erase the skipped items from memory. 

Display colors need to be standardized across these checklist devices in accordance with current 
industry standards and FAR 25.1322 which prescribes color coding for warning, caution, and 
advisory lights in the cockpit. For example, one major manufacturer uses the color green to 
indicate accomplished items while the other manufacturer uses the same color to indicate non- 
accomplished items. 

Experiments with computer-aided checklist. Rouse and Rouse (1980) conducted an 
experiment to evaluate the use of an on-board computer for checklist presentation. The computer 
retrieved the procedures from its database and provided a pointer by dimming the accomplished 
items on the screen. The results indicated that this computer-aided procedure is superior to a paper 
checklist in reduction of both error rate and execution time. 

In a second experiment Rouse, Rouse, and Hammer (1982) compared a computer-aided checklist 
(which was part of the feedback loop) with a paper checklist. The use of normal and abnormal 
procedures was evaluated in a flight simulator. The computer-aided checklist was significantly 
lower in error-rate, while the paper checklist was significantly faster in execution time. Rouse et al. 
(1982) explained that the slower execution time for the computer-aided checklist "is likely to be 
eliminated by training and/or redesign of the keyboard" (p. 462). 

Computer-aided checklist within the feed back loop. The Airbus A- 320 utilizes its 
Electronic Centralized Aircraft Monitoring (ECAM) computers to aid the crew in configuring the 
plane for takeoff and landing. The critical items in the TAKEOFF and LANDING task-checklist 
appear on the CRT prior to those segments. In addition, the computer displays a pointer system to 
indicate accomplished items and informs the pilot about the status of each checklist item. Once 
items are accomplished, the computer uses its logic to inhibit these items and clear the screen. 

Although the checklist portion of the ECAM can be programmed to include all of the normal 
task-checklists. Airbus Industrie has opted to program only the TAKEOFF and LANDING 
task-checklists. As a result, the rest of the normal task-checklists must be conducted from a paper 
card. 


Display and pointer checklist 
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Figure 7. Courtesy of Honeywell-Sperry Commercial Flight Systems Group 



Monitoring computers and checklist use. One of the important topics regarding usage of 
checklists in combination with a monitoring computer, is the logic and usage of the recall 
function. It requires human factors design as well as user-training to reduce the likelihood of losing 
skipped items, or not retrieving information that exists in the computer but is not displayed to the 
pilot because of screen limitations. One ASRS report describes a misconception of the recall 
function: 

Departing San Diego we were informed by the tower that someone had called on ground 
frequency to say that it appeared we had a hatch open on the aircraft. As the aircraft was not 
pressurizing, I leveled off and the captain recalled information on the El CAS <Boeing s 
Engine Indication and Crew Alerting System>. The display now informed us of two open 
doors.... I had erased all information from the CRT while doing the fire warning test 
section of the BEFORE ENGINE START checklist, as I was trained to do, to determine 
that you have a valid fire test and to have a clean CRT for engine start. I did not manually 
recall advisory information to the CRT, as I had been led to believe, during my training, 
that the electric power change-over from ground/APU to engine driven generators was 
electronically sensed and that my messages that applied would be automatically displayed. 
My understanding was that the manual recall was merely a backup if the automatic function 
was in error.. ..An informal survey among my peer group flying this airplane leads me to 
believe that I am not alone in this misconception... (ASRS #54596) 

The advantages of a computer-aided checklist, regardless of whether or not it is integrated to the 
feedback loop, are quite obvious. The device aids the pilot by providing a pointer, storing skipped 
items, and eliminating the need to occupy one hand in holding the checklist card. However, there 
are also some disadvantages to be considered: 

1 . Limited monitor size. 

2 . Non-adjustable distance of the CRT from the operator’s eyes, this factor becomes more 
noticeable as the pilot’s accommodation decreases with age. 

3 . Inferior alphanumeric quality (compared to print on a card). 

The high cost of updating the checklist text. 
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3. CHECKLIST CONCEPTS 

Different checklist concepts are used by the airlines, and they have a marked effect on checklist 
design. We will discuss these concepts and methods in the following section. In addition, we will 
detail the process of certifying the checklist according to FAA regulations. 


3. 1 . "PHILOSOPHY OF USE" 

The various ways of conducting a checklist are not only limited to the device in use, but they also 
pertain to the concept of using a checklist— sometimes referred to as the checklist philosophy ot 
use." This philosophy varies between airffame manufacturers, officials of regulatory agencies, 
and airlines. 

2LLJL The Airline "Culture 1 * 

In most cases, the checklist philosophy-of-use is the outgrowth of the company s corporate 
"culture." This term includes many factors that contribute to the overall operational concept or the 
organization, including management style, supervision concepts, delegation of responsibilities m 
the chain of command, punitive actions, etc. Airline culture may in part be formed by the region or 
the United States from which the company grew. The personalities of the founding fathers of the 
company may also exert a lasting influence, long after they have departed. Other factors influenced 
by "corporate culture" involve traditional methods of operation, pre-defined work policies, and 
management priorities. The airline's culture is an important factor because it is mirrored in the 
manner in which flight management and training departments establish, direct, and oversee flight 
operations and related procedures. 

3.1.2. Redundancy 

Redundancy is the concept behind many aviation systems and subsystems (Nagel, 1988, Sears, 
1986). It is an important factor in checklist philosophy of use. Sanders and McCormick (1987) 
state that "because humans are often the weak link in the system, it is common to see human- 
machine systems designed to provide parallel redundancy" (p. 18). A similar principle of backup 
and redundancy is applied in the checklist procedure. There are two types of redundancies available 
for this procedure. The first is the redundancy between the initial configuration of the aircraft and 
the use of the checklist as a backup only (configuration redundancy). The second is the redundancy 
between the two or three pilots supervising one another while conducting the checklist (mutual 
redundancy). 

3.1.3. The Device 

It is apparent that the type of device in use is a factor in the checklist philosophy-of-use. The 
different checklist devices are detailed in Section 2.2. However, since almost all commercial 
carriers in the U.S. use the paper checklist, the following discussion will focus on this particular 
device and its different philosophies of use. 

3.1.4. The Method 

There are two main methods of conducting the checklist— the challenge-response and the do- 
list." Each one is the outcome of a different operational philosophy . Nevertheless, there is no 
absolute boundary for each method, and variations as well as combinations of these methods exist. 



Challenge-Response. In this method, which can be more accurately termed "challenge- 
verification-response," the checklist is a backup for the initial configuration of the plane. Here, the 
pilots use their memory and other techniques to configure the plane. After completing the initial 
configuration, the pilots use the checklist to verify that several critical items have been correctly 
accomplished. The process of conducting this checklist method is as follows: Pilot A calls the 
checklist item from the printed list; pilot-B and pilot- A together verify that the item is set properly; 
and then pilot-B calls the verified status of the item, and so on. Hence, both the configuration and 
mutual redundancies are employed in this method. In three-pilot aircraft, the flight engineer almost 
always assumes the pilot challenging role and reads a large portion of the checklist. 

Do-list. This method can be better termed "call-do-response." In this method, the 
checklist is used to "lead" and direct the pilot in configuring the aircraft using a step-by-step, "cook 
book" approach. Therefore, the configuration redundancy employed in the challenge-response 
method is eliminated here. The process of conducting this method is as follows: Pilot-A calls for an 
item; pilot-B positions or sets the item to the correct position, and then announces the new status of 
the item (e.g., "flaps-15"). Once the item is accomplished, the next item is read and so on. In a 
three-pilot crew, the flight engineer is included. 

Most do-list checklists of transport aircraft are generally very detailed and time-consuming. In 
addition, due to the elimination of the configuration redundancy, a mistake can easily pass 
unnoticed once the sequence is interrupted The philosophy behind this method is very stringent in 
dictating the precise way of configuring the airplane. Not surprisingly, this method is widely used 
by the military in combination with the scroll checklist for several types of transport aircraft. 

Combined. Several checklists used in the airline industry employ a combination of 
challenge-response and do-list methods. In most cases, the challenge- respo nse is the dominant 
method, while the do-list method is used for the ENGINE START, AFTER TAKEOFF, and for 
the SECURE task-checklists. 

Summarizing, the do-list and the challenge-response methods are almost standard in the airline 
industry. Most carriers, however, favor the challenge-response or the combined method. Table A 
shows the tabulation of checklist methods used by a sample of 20 airlines worldwide (Boeing, 
1989). 


Tabulation of different checklist methods used by 20 airlines 


Method 

Number of 
Operators 

Challenge and 

12 

response 

Delist 

1 

Combined 

7 

Total 

20 


Table A. From Boeing, 1989 




3. 1 .5. The Items of the List 

The matter of which items should be presented on the checklist is a cardinal question in checklist 
philosophy. Some will argue that most of the configuration items required in operating the aircraft 
must be presented in the checklist. Others will argue that since the checklist is a redundant task, 
only the critical and most important items should be presented on the checklist. This statement 
leads to another controversy-- which items are critical and important enough to be registered on the 
checklist. An example would be the "shoulder harness" check. Some argue that this is a critical 
item that should be listed on the checklist. Others hold that the chances of an accident propagating 
directly from such an omission are very slim, that the use of the shoulder harness is second nature 
to the pilot, and therefore there is no need to include this non-critical item on the checklist. 

Taking a systems engineering approach, the first argument appears correct. If the checklist 
procedure is supposed to verify that the plane is configured correctly, then all items involved 
should be checked. This approach can lead to a very long and detailed checklist. The opponents of 
the above approach argue that a long and detailed checklist is no guarantee of absolute safety. 
Indeed, it carries the risk that some pilots might choose not to use the checklist or conduct it poorly 
because of its length. 

From a human performance and psychological approach, the second argument seems more 
appropriate. If the procedure is to be used by humans in the cockpit, the checklist should 
accommodate human nature-its capabilities as well as its limitations. However, this approach may 
also subsequently produce problems. The plane may not be configured correctly in the setup 
phase, but this will pass unnoticed. Therefore, the philosophy of any checklist must detail what 
type of items should be presented as well as what kind of items should not be presented. It must be 
noted that legal departments can intrude on these checklist decisions, and their solution would 
always be to include the questionable items. 

There is no method by which flight management can assure itself that the crew will configure the 
aircraft properly for the flight, except to trust the crew's training, experience, and discipline. The 
FAA, however, mandates the use of a checklist procedure to confirm this. In some cases, it 
appeared that flight management required that the pilots conduct a very long checklist only to 
assure itself that every single configuration item is done (critical as well as not so critical ones), and 
in order to meet legal requirements. 

3.1.6. Type of Operation 

There are conspicuous differences between long-haul and short-haul operation with respect to 
checklist philosophy and use. 

Short-haul. Pilots who fly short flight segments perform the flight checklists as much as 
3-10 times per day and as many as 10-30 times on a typical trip. Therefore, a requirement to 
conduct a long and meticulous checklist for each flight may lead some to deviate from the 
prescribed procedures, performing only what he/she perceives as the critical items ("killer items" as 
some call them). In cases where a checklist procedure is incompatible with the operational 
environment, the checklist may not be performed as prescribed in the SOP. For example, one 
company's DC-9-50 checklist (a short/medium range type aircraft) contained 81 check items for the 
ENGINE START, TAXI and TAKEOFF task-checklists. Not surprisingly, pilots from this 
company expressed concern about poor checklist discipline in short-haul operations. 
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T^ng-haul. In this type of operation, the reverse condition occurs. Pilots who fly long 
routes perform the flight checklists only once or twice a day, and as little as 4-8 times per trip. In 
addition, the realization of the importance of the checklist is much higher in over-water operations 
because of the crew's limited ability to divert to alternate airports following an omission or 
malfunction. The interviews indicated that crew members that were engaged in long-haul 
operations were less resistant to a long and detailed checklist. It may provide their last chance to 

make sure things are right. 


3.1.7. Automation 

The use of computers on the flight deck allows for automated monitoring of flight status and more 
effective feedback to the pilot (Wiener, 1989). On-board computers can alert the crew when certain 
configuration items have not been accomplished and display the actual status of the P™ c * ® c . 
computer can be used to verify that certain items have been accomplished ( recall of the EICAb in 
Boeing 767/757 models) and thereby substantially reduce the number of items on the checklist. 


This has led airplane manufacturers and operators to alter their checklist philosophies to coincide 
with the capability of the system monitoring computers. Boeing's B-757 normal checklist 
philosophy states that 


Normal checklists are used to verify that certain critical procedural steps have been 

accomplished. Only procedural steps which, if omitted, would have a direct and adverse 
impact on normal operation are included. Items annunciated by the Crew Alerting System 
(EICAS) are not included. (Boeing B-757 flight manual, 1985) 


Not all carriers employing the B-757 agree with the above statement, and some require their flight 
crews to check items even though these items are prompted by the on-board computer. The 
concern here is the level of reliance on automation. The advocates of complete use of computer 
capabilities argue that the computer is far more reliable than the human in monitoring, reduces 
workload, and decreases checklist distractions. Opponents argue that regardless of software and 
hardware redundancies, computers fail, and reliance on automation might lead to an 
incident/accident. As examples, they point to the failures of the Central Aural Warning System 
(CAWS) of the Northwest flight 255 (MD-80) and the takeoff configuration warning of the Delta 
Air Lines flight 1 141 (B-727). These systems were designed to warn the pilots of improper takeoff 
configuration. However, due to reasons unknown and component malfunction, both systems 
failed. 


The differences in automation philosophy between airlines flying the B-757 can be scrutmized by 
comparing the number of items in the ground phase (ENGINE START through BEFORE 
TAKEOFF) checklist of each company. One company’s checklist includes 50 items that the pilots 
are required to check in this phase, while another company's checklist has only 13 items (the 
authors omitted ACARS items and other unique items in this comparison). 


3. 1 .8. Recommended Philosophy 

Apart from the airframe-manufacturers' and airlines' philosophies, the Air Transport Association 
(AT A), an industry-wide trade organization, has also stated its recommended checklist philosophy. 

Checklists should contain, in abbreviated form, all the information required by the trained 
flight crew to operate the airplane in most normal and non-normal situations. Nornial 
checklists should be organized by segments of flight. The checklist should contain the 
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minimum cues required for the trained crew-member to verify that the appropriate actions 
have been completed. Only procedural steps which, if omitted have direct and adverse 
impact on normal operations, are included. Items annunciated by crew alerting systems are 
not included. (ATA, 1986, p. 46) 

It is interesting to note the similarity of this philosophy statement to the one advocated by Boeing 
regarding length of checklist and level of reliance on automation. 

In sum, it is very tempting for any flight management or outside entity (media, legal, etc.) to argue 
that all configuration tasks performed in the cockpit are important and, therefore, almost all 
configuration items should be presented and checked with the checklist procedure. On the other 
hand, the minimal checklist might fit more easily into the operational environment and human 
limitations. However, a consensus can never be reached as to which list of items are important 
enough to be included on the checklist. Furthermore, the success of this minimal item approach is 
difficult to measure in everyday line operation, and is even harder to defend after an occurrence of 
a checklist-related accident. 


12* CERTIFICATION OF CHECKLISTS. 

The certification process of each checklist is unique because each airline's checklist is tailor-made 
to suit its philosophy. In addition, many forces converge to shape the checklist before it is actually 
operated by line pilots. We will follow this process from the airframe manufacturer to the daily use 
of the aircraft by the carrier. 

3.2. 1 . The Airframe Manufacturer 

The process begins with the airframe manufacturer who designs the aircraft and determines its 
operational concept. This operational concept is the source from which the checklist is derived. 
After the first checklist is designed, it passes through a process of modification and later fine 
tuning" in flight testing. 

The FAA certifies the plane under Federal Aviation Regulation (FAR) Part 25, which "prescribes 
airworthiness standards for the issue of type certificates, and changes for those certificates, for 
transport category airplanes." In addition, "...information and instructions regarding the 
peculiarities of normal operations (including starting and warming the engines, taxiing, operations 
of wing flaps, landing gear, and the automatic pilot)" are also certified under this regulation (FAR 
Part 25.1 and 25.1585). 

After complying with FAR Part 25 regulations the new aircraft is certified by the FAA to fly. 
However, it is not certified yet to be operated by the customers-the airlines. 

3.2.2. The Airline 

Once the plane is sold to the airline, the second certification process takes place. This certification 
process (FAR Part 121-large commercial transports) is aimed at certifying the carrier to operate the 
airplane. In order to prepare for this certification, the fleet manager (or the checklist designer) takes 
the manufacturer's previously approved procedures and modifies them to coincide with the 
operational concepts and checklist philosophy of the airline. 
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The FAR Part 121 certification process is conducted by the Principal Operations Inspector (POI) 
assigned to the carrier. This individual and his staff are the FAA's representatives to the earners. 
They are responsible for initial certification, follow-on procedural changes, and regulatory 
oversight of the carrier. 


FAR 121.315 (see p. 2) is the only statement pertaining to the use of checklists by air transport 
carriers as mandated by the federal government. It does not comment about the concept, method, 
type, philosophy, or presentation of the checklist. Rather, it states the need for such a list, and 
mandates its existence in the cockpit. It leaves almost all aspects of checklist construction for e 
airline to design and the POI to approve or disapprove. 


For FAR Part 135 operators (regional carriers or commuter airlines), the checklist must agree only 
with the FAA-approved training program for the applicable commuter airline. There is no formal 
approval of checklist changes under this FAR; however, the duty to verifythat the checklist agrees 
with the training program also lies with the POI assigned to the airline (NTSB, 1988b). 


With the vague and non-specific nature of FAR 121.315, and with no FAA internal guidelines for 
checklist design criteria, the POI has almost total authority over the approval of the checklist. He or 
she may approve or dismiss it according to his or her own philosophy of checklist usage. 


3.2.3. Modifications 

Once the checklist is approved by the POI under FAR Part 121, it can be used for flight operations. 
However, changes and modifications do not stop here; they continue throughout the life of the 
aircraft in the company. The different sources of changes and modifications and their effect on 
checklist design are discussed below. 

Manufacturer. The first type of checklist changes are those changes initiated by the 
manufacturer. These changes are mainly due to knowledge gained about the airplane in line 
operations (as opposed to flight testing). Although some changes in the checklist following the 
introduction of a new airplane are expected, frequent changes can have a negative effect on the 
realization of the checklist's importance by the flight crews. This type of problem was cited by the 
Safety Board (NTSB, 1988b) following an investigation of the Air New Orleans, British 
Aerospace (BAe) J-31 commuter accident mentioned in the introduction to this report: 

...Frequent revisions of checklists for newly-acquired aircraft are understandable, but the 
fact that this <normal checklist> had been changed seven <italics added> times between 
January and May 1987 suggests to the Safety Board that its original design <BAe> and 
approval <FAA> may have been inadequate and may have caused confusion among flight 
crews, (p. 22) 

Conceptual changes. The second type of checklist modification arises from conceptual 
changes. For example, one airline that performs mainly short-haul operations has adapted the 
B-757 checklist concept (minimal checklist of only critical items) to its fleet of 
B-737s (100/200/300). These changes have led to a new philosophy of checklist usage now 
advocated by the airline's training department, and a reduction in the number of items on the 
checklist. 

New equipment. The introduction of new equipment in the cockpit requires revisions of 
the checklist (e.g., TCAS, ACARS). Again, this is not simply a matter of adding the items related 
to the new hardware, but also of determining their place in the checklist sequence. When a 
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company makes a significant change to the checklist, such action ...should be coordinated with 
the manufacturer to ensure that proposed operational procedures are adequately evaluated <by the 
manufacturers- " (Sears, 1989). 

New regulations. Changes in regulations, even those seemingly remote from the cockpit, 
can affect checklist design and use. The regulation prohibiting smoking on flights of less than six 
hours is an example. Prior to this regulation, the "no smoking sign was used to signal the cabin 
crew to prepare for landing. With the passage of the new non-smoking regulation, changes were 
required in the procedure and checklist. 

Operational. These changes are due to problems occurring while pilots perform the 
checklist and associated tasks in daily line operation. The decision to make these changes must be 
applied with caution, because too many and too frequent changes will tend to affect flight crews 
checklist performance adversely. 

Another difficulty in carrying out an operational change is the frequency of occurrence of a 
particular problem. Some managers, upon being made aware of a pilot making an error that is 
related to configuration, may feel that since one pilot could make the mistake, then the only way to 
prevent others from making the same mistake is to add new provisions to the checklist. For 
example, several reports to the FAA stated that on two occasions, B-757 s belonging to one earner 
landed with the flight attendants standing in the aisle. The company's procedure was to use the 
automatic function of the "no smoking" sign (activated when the landing gear is lowered) to alert 
the flight attendants to prepare the cabin for landing. Subsequently, the POI demanded that this 
item be done manually and added it to the BEFORE LANDING checklist. Therefore, a simple 
automatic feature designed to relieve the flight crew during a high workload segment of the flight 
was added to the checklist, thence to be performed manually. 

The checklist is constantly changed and modified, making it a dynamic procedure. However, in 
some cases its unique role in the operation of the aircraft is incorrectly used: it becomes a dumping 
site," to resolve discipline problems, and/or to show management and regulating officials that a 
specific problem is settled. By placing these type of item(s) on the checklist, immediate problems 
may be resolved, but the importance attached to the procedure by the pilots is reduced, leading to 
additional and possibly even more severe problems. 


TT STANDARDIZATION OF CHECKLISTS 

Standardization of flight-deck procedures between different fleets is a factor that is part of the 
operational concept of the airline. It is obvious that a conscientious flight management will set a 
goal of minimizing the differences in operational procedures between fleets in order to aid pilots in 
transitioning from one aircraft type to another. Not surprisingly, 16 airlines out of 20 that were 
determined by Boeing to have a lower-than-average crew-caused accident rate have responded that 
"...document philosophy and format is the same for all airplane types operated" (Boeing, 1989). 

Although the benefit of this approach to flight safety is quite clear, over-emphasis of this issue can 
hamper the effectiveness of the checklist. This is apparent when a checklist sequence in the cockpit 
does not correspond to the location of items, or when items which are not significant in one aircraft 
are added for the sake of standardization with other aircraft. 

Standardized procedures which were common to cockpits of older generation aircraft (L-101 1, DC- 
10, B-747-200) are sometimes not compatible with newer generation cockpits such as the 
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B-757/767, MD-88, A-320, and B-747-400. An attempt to enforce standardized procedures that 
belong to older generation airplanes on the checklist procedure of a modem airplane may result in 
poor checklist design, and may lead to safety problems. 

Many of the new airplanes that fly today, and those that will fly in the next decade, are modem 
derivatives of older airframes (MD-88, and B-737/300/400 are derivatives of the DC-9 and 
B-737/1 00/200 respectively). However, these modem planes differ from their counterparts in 
power plants and cockpit systems (EFIS, FMC, etc.). There are many questions arising in the 
industry today as to how to consider these fleets that are very similar in appearance, yet very 
different in technology. Some opt to treat them as completely different airplanes, providing 
separate training programs and bidding processes ("separate status"). Others regard them as having 
equal status when the FAA grants a "common type rating" (e.g., DC-9 and MD-80). Similar 
problems arise in checklist design. The question is whether the checklist should be standardized 
between the derivatives, or each derivative should have its own checklist. One ASRS report 
illustrates the effect of this problem on flight operations, 

...During this segment of flight the flaps/slats handle was selected to 5 degrees while 15 
degrees was required. This improper selection was initially missed on the checklist, 
however it was corrected before the taxi checklist was called complete.. ..such positioning 
errors may be a result of the following: 

1) When the flight crew operate three series of aircraft within a single aircraft type rating, 
with variations on each series. 

2) Operating of these aircraft on a daily basis. 

3) Transfer of crew-members from aircraft to aircraft on a rapid fast moving flight 
schedule. 

Such errors may be reduced with some of the following suggestions: A type of regulatory 
environment for cockpit standards.... Careful review of the cockpit checklist corrected the 
flap setting problem, However, such may not always be the case... (ASRS #92957) 

Management and training departments can try to soften this effect, as the following example 
concerning derivatives illustrates. The weather radar of the MD-80 (a derivative of the DC-9), 
operates on low transmission power, and there is no need to shut off the radar on the ramp during 
intermediate stops. However, the DC-9 radar has a high power transmission and it is required that 
the radar be shut off on the ramp for the safety of ground crews. While both planes were in 
operation in one company and under the same "status," flight management decided to require flight 
crews of the MD-80 to shut off the radar during intermediate stops, and thereby reduce the 
probability of a MD-80 pilot forgetting to shut off the radar while flying a DC-9. 


3.4. TWO/THREE-PILOT COCKPIT 

The prevalence of two-pilot cockpit airplanes has a substantial effect on the philosophy of checklist 
usage. Airline checklist philosophies have evolved during the era of two- and three-pilot cockpits. 
Consequently, the traditional paper checklist and the challenge-response method (mutual 
redundancy) are the outcome of these operational environments. In the three-pilot cockpit the flight 
engineer assumes a major role in the checklist procedure and its management. This non-piloting 
crew member is responsible for working the plane's systems, and hence is less occupied with 
outside actions such as ATC instructions, ground crew communication, taxiing, etc. Situated 
behind the pilots and at hand's reach from most systems controls, the flight engineer is able to be 
the conductor of this procedure, and serve as its quality control inspector. In the two-pilot cockpit. 
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the position of the flight engineer was eliminated and his duties were absorbed either by cockpit 
automation or by the two pilots (McLucas et al., 1981; Wiener and Curry, 1980). This 
transformation necessitated changes in task assignments, cockpit management, and checklist 
philosophy-of-use. 

To manage the new situation, several airlines relaxed the checklist in certain flight segments by 
allowing the pilot not manipulating the controls to challenge himself and respond to himself. 1 hus, 
these airlines have eliminated the mutual redundancy that is part of the traditional checklist 
procedure. Several flight management officials that were interviewed argued that the quality of this 
procedure (paper checklist and challenge-response method) in the two-pilot cockpit is below 
expected standards. 
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4. AIRLINE MERGERS AND ACQUISITIONS 

In the 12 years since the Airline Deregulation Act of 1978, numerous companies have been taken 
over, merged, or sold to other companies. The amalgamation of two (or more) organizations with 
different operating methods, and the amalgamation of pilot groups, pose many problems to the 
management of the new airline. Some of these problems have an effect on checklist use and 
concept and therefore will be discussed below. 

After a merger or a takeover, an adjustment period takes place. During this period the acquiring 
company inspects the operational procedures of the acquired company, and initiates a program to 
standardize the procedures and checklists of the acquired airline. The new standardized 
procedures are, of course, based on the philosophy of the acquiring company, which is sometimes 
very different from the philosophy of the acquired company. The acquiring company can be 
expected to assume that its philosophy is correct and that little can be learned from the acquired 
company. An ASRS report illustrates the potential impact of such changes on line operations! 

Our company was bought by a competitor about a year and a half ago. The new airline has 
specific policies, procedures and philosophies that differ from the former airline that I 
believe contributed significantly to this incident. At the former airline, the F/O's were 
taught to automatically extend flaps and slats when the aircraft was saluted away by the 
ramp agent. The new airline allows slats/flaps extension only after the aircraft is away from 
the ramp and upon command of the captain....Upon reaching the outbound taxiway and 
due to a long line of aircraft waiting I shut down number one engine to save fuel. We 
couldn’t complete the taxi checklist (which includes flaps/slats) because company 
procedures prohibit accomplishing this task while taxiing on one engine. My former airline 
used a single engine taxi before takeoff checklist, which assured extension of 
flaps/slats.... Per company procedures I called for the before takeoff checklist rolling onto 
the takeoff runway. I knew the next landing aircraft was rapidly approaching the 
runway....I advanced the power levers and immediately the voice warning system said 
"flaps/slats." At this point it was obvious we had not accomplished the taxi checklist. For 
purpose of standardization, the new airline did not accept any of the former airline 
operating policies or procedures (including checklist), even though there were 22 years of 
operation with the former airline~and many checklist improvements over that time to assure 
the safety of operations.. ..There is an obvious need for proper authorities to consider 
factors such as past practices before approving checklists on airlines that have been merged 
or bought out... (ASRS, #91277) 

During a merger or an acquisition of an airline, the personnel of the acquired company may go 
through a period of emotional anxiety that affects their performance. In particular, flight crews may 
go through a painful process involving financial uncertainty, base and route changes, and collapse 
of existing seniority structure (Lautman and Gallimore, 1988). 

...went to crew room for flight release, and got the new information concerning company 
buy-out. Upset again like the last four years. Two choices: sign up or no job....I began the 
checklist but was interrupted by passing on the buy-out news to the flight attendants. First 
officer and I continued the discussion and did not finish the last item on the checklist. 

When I looked at the fuel gauges and totalizer I must have seen 9.9 instead of the required 
19.0 cthousand pounds>....Over ALS VOR, our first fuel check point, fuel gauges were 
showing approximate 4000 pound total.... First officer looked for the fuel slip. It was not 
on board.... Contributing factors: Over the last four years our airline has gone through 
many changes. Upstart airlines, scab pilots, route and cities dropped, over half our aircraft 
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sold and ESOP <Employee Stock Ownership Plan> failed even with 50 per cent pay cuts 
and working concessions. ...Over these years none of the employees knew if the next week 
we will have jobs. At the beginning of this trip first officer even mentioned if anything 
(accident) were to happen it would be between now and the final outcome of the buy-out... 
(ASRS, #55218) 

In addition to the emotional anxiety, flight crews must also adapt to new operational philosophies, 
procedures, work rules and regulations. The combination of all the above factors coupled with 
resistance to change manifest themselves in poor group morale and anger towards the new 
company. In some cases this anger is displaced and transferred away from the source to other 
avenues. Displacement of feelings is defined by the American Psychiatric Association glossary as 
"a defense mechanism, operating unconsciously, in which emotions, ideas, or wishes are 
transferred from their original object to a more acceptable substitute; often used to allay anxiety 
(Werner, Campbell, Frazier, Stone, and Edgerton, 1984, p. 30). Based on the pilot interviews, the 
authors believe that some of this anger becomes unconsciously transferred to the normal checklist 
for several reasons: 

1 . The normal checklist is an ordinary and routinely performed procedure. 

2. The normal checklist is a redundant procedure, making it appear "safe" to misuse. 

3 . It is mandated by the company; however, no provisions for on-site enforcement are visible. 


Mergers and acquisitions generate significant problems that have a profound effect on flight safety 
in general, and use of checklists in particular. These difficulties "...require anticipation and 
alertness on part of management to minimize the impact on safety," and much sensitivity from the 
acquiring operational management to accommodate these difficulties (Lautman and Gallimore, 


1988, p. 8). 



5. LINE OBSERVATIONS OF CHECKLIST PERFORMANCE 

Until now we have discussed the different checklist devices, methods and concepts. In this section 
we will discuss the process of conducting the checklist in line operations. This process can usually 
be divided into three steps. The first step is checklist initiation, the second is the routine of calls 
and responses, and the third is the completion of the task-checklist. In addition, the effects of 
distraction will be discussed. 


SI INITIATION 

The initiation of the checklist requires the pilot flying (PF) or the captain to judge when to call for 
the task-checklist, and to recall if previous checklists have been done and properly completed. This 
process, when coupled with high workload, stress, and schedule pressures can lead to checklist 

initiation errors. 

Many pilots use internal as well as external cockpit cues to aid them in initiating the checklist. For 
example, the BEFORE START checklist can be cued with closing of passenger doors; the TAXI 
checklist after receiving the taxi clearance; the BEFORE TAKEOFF checklist by reaching the hold 
line before the runway; the DESCENT checklist at a transitioning altitude of 18,000 feet. Checklist 
cues are usually not part of the SOP; rather, they are more a personal technique among pilots. 
There are some problems with these techniques: they are not always present or applicable, and if 
pilots are occupied with other tasks, cues can pass unnoticed. 

During the investigation of the Northwest Flight 255 accident, testimony from other Northwest 
Airlines pilots indicated that they usually complete the TAXI checklist within 1-2 minutes after the 
plane started to move on the ramp. However, due to several interruptions in the checklist progress 
(new weather information, checking aircraft and runway data), the TAXI checklist for flight 255 
was not completed within the first minutes of the taxi. "By this time the airplane's location on the 
airport was such that the external cues and references available to the flight crew were not those 
normally associated with the initiation of the TAXI checklist at Detroit-Metro (NTSB, 1988a, p. 
58). In other words, by the time the distracting tasks were finally completed, the regular external 
cues had vanished. 


5 2 CHALLENGE-RESPONSE 

In this section the authors will describe different checklist use behaviors, the problems associated 
with each and their effects on checklist performance. Most of this information was gathered during 
the field studies, and the rest from line pilot interviews. 

5.2.1. Memory-Guided Checklist 

In several instances during night operation, the checklist card was drawn out of the slot (above the 
glare shield), but no light was turned on to allow reading. Consequently, the checklist was 
performed from memory. A quite similar habit was observed in both day and night operation: the 
pilot would stretch his hand out and touch the checklist-card situated on the glare shield but would 
not draw the checklist out of its slot. It is interesting to note that pilots had a habit pattern of 
associating a motor response with the checklist procedure. Nevertheless, the card was not drawn 
from the slot and the checklist was read from memory. On another flight, we observed a flight 
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engineer in a B-727 run the entire checklist for a trip from memory, with his paper checklist placed 
in a crack in the edge of his panel. 

5.2.2. Verification 

In some cockpits, the task of verification was left only to the pilot responding to the checklist. The 
pilot challenging the checklist (PNF) read the checklist items but did not move his eyes away from 
the list to cross-check his partner. Therefore, the mutual supervision embedded m the checklist 
procedure was not utilized. 

Often, the pilot flying (PF) would answer with the proper response immediately when he heard the 
challenge call from the PNF, not verifying that the item called was set accordingly. This was 
clearly evident in high workload phases of flight such as during the approach (BEFORE 
LANDING checklist). In this case, the pilot must rely on his short-term memory to judge whether 
the checklist item was set correctly. Therefore, the configuration redundancy embedded in the 
procedure was lost. 

Several pilots who had the habit of not closely watching the item before responding to the 
challenge have added a personalized safeguard. The responding pilot would complete the entire 
"challenge-and-response" callouts, and only then focus on the items in order to verify that the 
responses he called before, did in fact portray the actual configuration of the airplane. It appears 
that the pilot did in fact sense the low quality of his mandatory checklist process, and therefore 
created this additional safeguard. Likewise, several pilots who were interviewed stated that they 
have their own checklist procedure which they perform from memory just pnor to takeoff. 
Nevertheless, relying on these memory techniques has some inherent hazards: 

1 . It is dependent on the availability of time after the quick completion of the checklist. 

2. It is vulnerable to distractions such as ATC communications, outside scan, starting an 
engine during TAXI segment, and more. 

3 . It is based on memory, and not on a step-by-step challenge-and-response. 

5.2.3. "Short-Cutting" the Checklist 

Several pilots deviated from the challenge-and-response method to a faster technique. This 
technique was to call several challenge items together in one "chunk," while the other pilot would 
reply with a series of chunked responses. This technique of conducting the checklist undermines 
the concept behind the step-by-step challenge-and-response procedure. It is also dependent on the 
pilot's short-term and long-term memory as to the order and completion of the checklist, which, in 
fact, is exactly what the checklist is supposed to prevent. Swain and Guttman (1983) found the 
same technique employed by nuclear power plant operators. They defined this non-standard ^ 
technique as "performing several steps and then checking them off all at once on the checklist 
(chap. 16, p. 2). 

When the normal checklists are lengthy, there was a tendency to actually perform the items while 
reading the checklist in an effort to overcome a laborious and time-consuming procedure. For 
example, lights, pitot heat, and transponder are usually toggle-type switches on the panels. The 
pilot would call the challenge from the checklist-and then position the item accordingly. However, 
by doing so, the crew lost the configuration redundancy imbedded in the checklist. While this short- 
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cutting may not always be related to the critical configuration items, it can easily migrate to items 
that are critical to the safety of the flight. 


5 3. COMPLETION 

The lack of indication that a task-checklist is fully completed is one of the handicaps of the paper 
checklist. The only safeguard here is a completion call, such as "The BEFORE START checklist is 
complete" which is made by the challenging pilot as he completes the checklist. 

The aircraft had arrived late at the gate in Chicago, crew change and this resulted in 
minimum turn-around time. We had not completed the BEFORE START checklist when 
the mechanic called for pushback. I first became aware of the oversight when the mechanic 
called, after the engines were started, that he was having difficulty disconnecting the tow- 
bar and he asked if the "A" hydraulic system was pressurized. Fortunately he was not 
injured....In order to preclude this from happening again, my personal procedure is to 
place the checklist in a obviously different position-on top of the radar screen....until the 
BEFORE START checklist is complete... (ASRS #47488) 

Some airlines write the completion call as the last item in each task-checklist, making the call itself 
the final checklist item. Some choose not to list this call in the checklist, but still require the pilots 
to make the completion call. A few other airlines disregard this call completely. 

The field study showed many cases where pilots (using a checklist without a written completion 
item) chose not to make this callout, or made a very faint (mumble) callout that probably was not 
heard by the other pilot. In these cases, it appears that the gesture of returning the checklist card to 
its place on top of the glare shield was the only notification of completion. However, if the PF is 
occupied with another task, he many not be able to observe this movement. 

The completion call is a redundant action. In most cases crew members know that the checklist is 
completed. However, this is the only reliable feedback available to indicate this. Furthermore, the 
statement that a specific checklist is complete provides a "cap" to the checklist process and enables 
all crew members to mentally move from the checklist to other areas of the operation with 
assurance of completion. 


5.4. DISTRACTIONS 

Monan (1979) conducted a study of distraction reports sent to the ASRS in order to determine the 
causes of distraction in the aviation system. He states that "one of the frequently occurring causes 
of hazardous events in air carrier operation is the human susceptibility to distractions" (p. 3). He 
argues that due to distractions, one airman is removed from the operational loop and thereby a vital 
cross-checking function is eliminated. The operation becomes vulnerable to any error committed 
during "the one-man show." 

Distractions and interruptions can "break" the checklist process and may result in a checklist error 
or omission. Conversely, the checklist process itself can be a distractor for other cockpit tasks and 
duties. Of the 169 air carrier distraction reports analyzed in Monan's study, 22 where labeled as 
distractions caused by checklist procedures (Table B). 
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Types of reported distractions 


Type of distraction 

Number of 
occurences 

1. Non-operational activities 

Paperwork 

7 

PA system 

12 

Conversation 

9 

Flight attendant 

11 

Company radio 

16 

2. Operational workload tasks 

Checklist 

22 

Malfunctions 

19 

Traffic watch 

16 

ATC communication 

6 

Radar monitoring 

12 

Studying approach plate 

14 

Looking for airport 

3 

New first officer 

10 

Fatigue 

10 

Miscellaneous 

2 

Total 

169 


Table B. From Monan, 1979, Table 1, p. 5 





Monan stated that there were two characteristics common to all 22 reports: 

1. Every report indicated that checklist accomplishment received cockpit priority over 
ATC requirements. 

2. The <normal> checklist activity was almost always going on at the same time other 
cockpit tasks were being performed: radar monitoring, minor malfunction, systems 
operation, traffic watch, etc. Checklist accomplishment became a cause for 
distraction not by itself but as part of cockpit workload. In the incidents reported, 
the workload became 'excessive' and 'time ran out before all tasks could be 
completed, (p. 10) 

Monan argues that "when accomplishment of several tasks merge into simultaneous activity 
through coincidence, poor planing, or urgency, then a 'distraction-due-to- workload may occur 
(p. 9). It is certainly difficult to anticipate and eliminate all the distractions due to urgency, 
however, we believe the designer of the checklist can decrease the probability of a checklist 
process interrupting other tasks (and vice versa) by reducing the length of a given checklist. 
Nevertheless, the designer of the checklist can do only so much; from that point forward it is up to 
the captain to manage the cockpit in such a way as to minimize distractions. 

5.4.1 . Airborne Phase 

In 1983, a Republic DC-9-82 (MD-82) had both engines stop while cruising at 35,000 feet, 20 
miles north of Bryce, Utah. During the emergency descent, the crew performed the emergency 
checklist and switched "on" all of the fuel boost pumps. Upon reaching 12,200 feet, both engines 
started and the crew diverted to Las Vegas, Nevada. The NTSB investigation revealed that the 
main fuel tank on each wing was empty while the center tank contained all the fuel needed to 
complete the flight safely. According to the airlines' DC-9-82 CLIMB checklist (challenge-and- 
response by the PNF only), center-tank fuel boost-pumps were to be switched on shortly after 
takeoff (Figure 8). During the takeoff, the autopilot heading- select knob came off, momentarily 
distracting both pilots and leading the captain to call for the CLIMB checklist "out of normal 
order." The first officer wrote: 

At that point and during a turn, the captain called for the "climb check." Because of our 
weight and my being new in the plane, I thought he wanted to turn with flaps and slats 
extended, so I proceeded with the climb check. I planned to go down the checklist to, but 
not including the hydraulic pumps, and then wait for the flap/slat retraction command. 
During the checklist, as I completed the ignition off <item>, he called for "flaps and slats 
retract." I then received a radio call to "change to departure" frequency. After flaps and 
slats were retracted and the radio frequency change, I continued with the checklist as I had 
planned. In retrospect, it appears that I may have left the center boost pump switches off. 
(NTSB, 1983) 
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Republic Airlines DC-9-80 checklist 


BEFORE START 


SEAT BELT *ON 

PITOT HEAT *CAPT 

WINDSHIELD HEAT *ON 

FUEL PUMPS * MAI NS ON 


AUX HYD PIJHP £ PRESS . . . . *OM £ CKD 
RAD I OS-ALT £ FLT DIR . . .**CKD £ SET 
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 
FUEL £ OIL . . . (QUANTITIES). £ RESET 
BRAKES £ IGNITION .... AS REQ £ ON 
AFTER START 


ANNUNCIATOR CKO 

IGNITION . *OFF 

ELECTRICAL POWER *CKD 

APU AIR *AS REQ 

AIR CGNO PACKS *AUTO 

PNEU X-FEEO *1 CLOSED 


TRANS PUMP £ HYD SYS . . . . *ON £ CKO 
TAXI 


FLAPS *15° 

TRIM **SET £ CKO 

EPR £ IAS BUGS **SET £ X-CKO 

FLT INSTRUMENTS . . . .*a(HDG) £ SLVG 

ANT I -SKID FARMED 

CONTROLS £ EI.EV PWR . . . **FREE £ CKO 

DELAYED ENG START 

BRAKES £ IGNITION . . . .AS REQ £ ON 
DELAYED AFTER START 


ANNUNCIATOR CKD 

IGNITION *OFF 

ELECTRICAL POWER *CKD 

APU AIR *OFF 

AIR COND PACKS *AU1J 


ENG ANTI -ICE £ FUEL HEAT ... AS REQ 


PNEU X-FEEDS . ^CLOSED 

APU *OFF 





REPUBLIC 


AIRLINES , INC. 


DC-9-80 

BEFORE TAXI OFF 

FLT ATTENDANT SIGNAL .... *2 BELLS 


TRANSPONDER . . . , 


*ON 

FLAPS 

. *i 

[SETTING) £ CKD 

ANNUNCIATOR . . . 


CKD 

IGNITION 

. . 

...... ON 

CLIMf; 


NO SMOKE 

• • 

*OFF 

IGNITION 

• • 

. . . . *AS REQ 

FUEL PUMPS . . . . 

• • 

*CKD 

CABJN PRESSURE . . 

• • 

*CKD 

A TR. COND AUTO SHUT 

OFF 

*OVRD 

HY^AUUC PUMPS . 

9 

. . . *OFF £ LOW 

FLAP' T.O. SEL . . 

9 

. . . . *STOWEO 

IN RANCI 


ALTIMETERS . . . 

*a SETTING) £ X-CKD 

EPR £ IAS BUGS . . 

• • 

. **SET £ X-CKD 

SEAT BELTS . . . . 

• • 

*ON 

CABIN ALTITUDE . . 

• • 


HYDRAULIC PUMPS . 

• 

. . .*ON £ HIGH 

BEFORE 

LA 41 

ING 

NO SMOKE 

• « 

*ON 

IGNITION 

• • 

*ON 

FUEL SYSTEM . . . 

. * SET FOR LANOING 

AIR COND AUTO SHUT 

OF 

* AUTO 

GEAR 

* • 

... *3 GREEN 

SPOILERS 

9 • 


AUTO BRAKES . . . 

• 

. . . . *AS REQ 


Figure 8. From NTSB, 1983 



5.4.2. Ground Phase 

When the aircraft is either at the gate or on the ramp, the checklist process sometimes needs to be 
halted to await certain conditions that are under the responsibility and control of external entities 
(waiting for fuel, waiting for cargo door to be closed, maintenance, etc.). One ASRS report details 
such an interruption of checklist sequence: 

...Beset by schedule delay and distracted by a jumpseat rider and a ramp-agent, the 
ENGINE START checklist was interrupted; the flight was "standing by for fuel." 
...inadvertently missed the only reference to fuel on any of our checklists. During taxi, I 
did glance at the fuel gauges, but since it was not in response to a checklist, I noted only 
that the gauges looked "about right." The next time I glanced at the gauges we were at 
cruise.. ..This time they didn't look "about right" and I immediately realized what had 
happened... (ASRS CALLBACK #115, 1988) 

Checklist holds. Several companies have a procedure to manage this situation. They 
require that during an interruption that leads to a stop in the checklist sequence, the pilot should call 
"hold on... <item>." Nevertheless, most pilots interviewed said that in practice only very rarely 
would they use this callout. A similar finding (not using the mandated "hold" call) was also 
reported by Linde and Goguen (1987) in their study of checklist interruption in a B-707 Line 
Oriented Flight Training (LOFT) sessions. On the other hand, several companies disregard this 
condition, and do not state how the pilot should behave upon an interruption or hold in the 
checklist sequence. When no special provisions are taken to mark the location of the hold, then the 
only place for retaining the hold-point is in the pilot's low-capacity short-term memory. However, 
this memory is highly susceptible to interference from communications (such as flight attendants, 
ATC, ATIS, etc.), and has a very limited retention time for stored information (Wickens and 
Flach, 1988). 

The step-by-step sequence of the checklist procedure may generate inexplicable errors because the 
position in the sequence must be indexed by some kind of a "pointer" in the working memory. 

This pointer, however, is easily displaced by any other form of activity during this sequence (D. E. 
Broadbent, private communication, March 14, 1989). One solution, mandated by a U.S. carrier, is 
to start the procedure from the beginning of the task-checklist following a hold. However, it is 
only reasonable to require this in cases where the appropriate checklist is relatively short. 

5.4.3. Maintenance 

All those agents involved in dispatching an aircraft, can disrupt the checklist process. However, as 
opposed to other agents who can only interfere or distract the crew while performing the checklist, 
maintenance personnel also have "hands on" access to configuration items. 

Often, maintenance personnel are under the same schedule pressures as the pilots for on-time 
departure. While working on a malfunction, mechanics sometimes need to alter the configuration 
of the aircraft and/or disable certain systems in order to perform their duties. But, under pressure, 
they may forget to reset these systems. 

...Arrived in ORD on a turnaround back to EWR.... Noted after landing the #3 thrust 
reversers lever was not quite flush with #1 and #2... .Maintenance began investigating. 
Maintenance worked on it until 15 minutes prior to our departure; determined it required a 
part, and placed the item on the open item list. The crew did the through-stop citalics 
added> portion of the checklist, then the BEFORE START prior to pushback.. ..During the 
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start I must have been distracted.. ..As my scan returned to the instruments, I expected to 
see approximately 20% N2 and some Nl....What I saw when I first looked at N2 was 
approximately 35% and accelerating. I looked down and saw the start lever in idle, then 
saw EGT peak at or slightly below die maximum allowable temperature for engine start on 
the ground....My first thought was that the first officer had raised the start lever as I was 
looking left without saying anything. I then noted that all 3 start levers were in the idle 
detent and it became clear that maintenance had placed them in that position during their 
work.. ..The full receiving checklist <first checklist of the day> calls for checking the start 
levers in the cut off position, which we did in when we received the plane in 
EWR....Through-stop checklist does not call for us to check the start levers in cut-off. Not 
expecting them to be in a different position from where we left them after the parking- 
checklist, all 3 crew members and obviously maintenance before us failed to catch that they 
were in idle... (ASRS #90128) 

It appears that cockpit management requires the captain to consider the work done by outside 
agents and assess the effects of this work on the proper configuration of the plane. He then can 
decide whether he wishes to conduct the "through-stop" checklist or conduct the lengthy preflight 
checklist even if the plane is at an intermediate station. This brings up another problem of cockpit 
documentation, the minimum equipment list (MEL). This topic is beyond the scope of this report, 
but suffice it to say that the MEL has suffered the same fate as the checklist: non-standardization 
from carrier-to-carrier, differences in philosophy, and the judgement of the individual POI. 


5.4.4. Inde xing the Checklist 

Swain and Guttman (1983), in their study of human reliability, analyzed Human Error 
Probabilities (HEP) for various tasks in nuclear power-plant operations. They have recognized that 
a checklist without a check-off provision "is more susceptible to errors of omission than a 
procedure with check-off provisions" (chap. 15, page 13). They reported that the estimated 
probability of error (per-item) for a checklist with no check-off provision was one in a hundred, 
while the probability of error for a checklist with some kind of a check-off provision was much 
lower, only three in a thousand. 

There are several personalized techniques that pilots employ to guard themselves from omitting a 
checklist item. The most common one seen by the authors was the habit of moving the left thumb 
along the left-justified checklist items. This technique uses the thumb as an index for the current 
item as well as a indicator once the checklist is interrupted. However, there are some problems 
with this technique: 

1 . In most checklists, the vertical spacing between the lines is too small to precisely identify 
the location of the thumb to a particular line. 

2 . If the checklist is interrupted for a length of time, it requires the pilot to hold the card with 
his thumb on the "hold" item throughout this period. 

3 . If the checklist is organized in two columns on the card, this technique will only be 
effective for the left-most column. 

Another technique is to use a grease pen to mark the location of a hold in the checklist and to "tick 
off* accomplished items. However, this technique becomes cumbersome in the cockpit operational 
environment, particularly at night. Some write the item on which the checklist was interrupted on a 
pad, and use this note when returning to complete the checklist. Several pilots from one company 
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stated that they extend the retractable magnetic compass in the B-737 cockpit to cue themselves of 
an abnormality associated with fuel. This technique is used to indicate that the plane was waiting 
for fuel at the gate, or while the engines were being cross-fed during flight. 

In sum, checklist distractions and interruptions lead to the following consequences: 

1 . Elimination of the vital cross-checking of the other crew member. 

2. Disruption of the sequential flow of the checklist. 

3 . Committing to memory the location of the interruption in the checklist sequence. 

The presentation and layout of the paper checklist does not allow visual indication of the point 
where a checklist was interrupted nor does it differentiate between accomplished and non- 
accomplished items. Therefore, the step-by-step sequence of conducting a checklist is the only 
available technique to control this procedure. However, as the above incidents show, ATC 
communication, ground personnel, maintenance and many other entities can easily break the 
delicate sequence of the checklist and leave the pilots with no safeguards against checklist 
omissions. 
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6. FACTORS INFLUENCING CHECKLIST INCIDENTS 

6. 1 . PSYCHOLOGICAL EFFECTS 

Several checklist-related incidents were the result of a situation in which a pilot thought he had set 
and checked a control properly, but actually had not. Some of the psychological factors influencing 
such mishaps will be discussed below. 

6.1.1. Perception and Mental Model s 

To perceive something is to be conscious of it and to pay attention to it. Perception is a dynamic 
process. It changes constantly depending on the physical stimuli and on the way in which the brain 
blends incoming information with information already stored in memory (Foley and Moray, 1987). 
Therefore, the mere existence of a physical stimulus obtained by a receptor (e.g., eyes) is not an 
absolute predictor of what the pilot will perceive and act upon while performing a task or checking 
the checklist items. 

When a certain task is performed repetitively in the same manner, operators become experienced 
with the task. In a sense, they actually create a "mental model" of the task. With experience, the 
shape of the model becomes more rigid, resulting in faster information processing, ability to divide 
attention, and consequently leading to a reduction in workload. In return, however, this model may 
adjust, or sometimes even override, the perception of physical stimuli coming from the receptors 
and bias the brain ("seeing what one is used to seeing"). One ASRS report details this effect: 

...During push-back from gate, tractor-bar broke because nose wheel steering hydraulic 
system was pressurized through the ground interconnect valve from "B" hydraulic system. 
History: the aircraft wing flaps were down and ground interconnect valve switch was open 
(non-standard), when the outbound S/O preflighted the aircraft. During the PRESTART 
checklist the S/O responded "ground interconnect-close," when switch was actually open. 
Prior to pushback captain's response to ground crew was "Interconnect— close," 

"hydraulic 'A' system depressurized." The ground interconnect switch was still open and 
"A" system was pressurizing... 

How can this happen? Two checks on the interconnect valve and "A" system pressure were 
missed! The only answer I have is-you look at this valve switch and pressure gauge on 
every departure check, and you may expect to see them closed and zero... (ASRS, #28177) 

Many of the pilots interviewed by the authors stated that at one time or another they had seen a 
checklist item in the improper status, yet they perceived it as being in the correct status and replied 
accordingly. For example, the flap handle is at the zero degree slot (physical stimulus), but the 
pilot perceives its location on the 5 degree position, and calls "flaps— 5," because he expects it to be 
there. This incorrect reply is based on numerous similar checks in which the flap handle was 
always in the proper setting during this stage in the checklist. Often, this phenomenon is coupled 
with unfavorable psychological and physical conditions such as time pressure, high workload, 
fatigue, noise, etc. Nevertheless, the result is a human failure. 

Most automobile drivers have had the experience of driving along a familiar route, and suddenly 
realizing that they have traveled some distance without being aware of it. The driver ceases 
consciously to process information for a significant length of time. "But the highly practiced skill 
of driving can be controlled by the output of the brain’s pattern analyzing mechanisms without 
conscious perception" (Foley and Moray, 1987). There was almost a consensus among the pilots 


38 


interviewed that many times checklist procedures become an automatic routine (’’sing-song" as 
some called it). The pilot would "run" the checklist, but the reply would be done from memory, 
and not based on the actual state of the item. The authors believe this is controlled by the output of 
the brain’s pattern analyzing mechanism, and that the check procedure is done without conscious 
perception. 

Preflight planning out of ABE showed we were right up to maximum weight for the 95 
degrees day. ...Further analysis showed that only a flap 5 departure could be made, and not 
flap 1 as normally done. Both pilots acknowledged this, but then both set flap 1 speeds on 
their airspeed bugs, set flap 1 on the flaps indicator and flaps 1 was acknowledged on the 
takeoff checklist. Halfway down the takeoff roll, I realized that the flaps were not set right, 
called out "flaps coming to 5 degrees" and moved the flap handle....The routine which 
developed turned into a very tough habit to break... (ASRS #55121) 

Similar problems in checklist performance may have occurred regarding the Delta Air Lines flight 
1141 accident in which "the flightcrew did not extend the airplane’s flaps or slats for takeoff 
(NTSB, 1989, p. 92). However, the proper checklist callouts for the to flap handle position, flap 
indicator dial, and slat extension (light) were made. During the analysis of this accident, the NTSB 
investigators measured the time delay between the second-officer's challenge ("flaps") and the first 
officer reply ("fifteen fifteen green light") as recorded in the cockpit voice recorder (CVR). They 
reported that "...the time between the checklist challenge and responses was less that one second, 
with little time to accomplish actions required to satisfy the proper response" (NTSB, 1989, 

p. 61). 

Reversion to older habits is another common phenomenon in aviation, and its extreme usually 
occurs following a pilot's transition from one plane to another (Rolfe, 1972). This can also affect 
checklist performance. For example, in 1987 an Air New Orleans BAe J-31 crashed after an 
aborted takeoff; the flight crew did not advance the RPM levers to 100% as per operating 
procedure and checklist. The captain and first officer had a limited amount of time on the aircraft 
(47 and 15 hours respectively), but both had considerable experience in a Beechcraft BE-99. The 
operating procedure and checklist of the BE-99 require that the RPM levers be set to takeoff 
position before taxiing. The BAe J-31 procedure requires that the same levers be set just prior to 
takeoff. Therefore, the item was the last on the BEFORE TAKEOFF checklist. The Board 
concluded that under urgency and stress imposed by the controller, "...they may have reverted 
back to recent habit patterns and began the takeoff believing that the RPM levers already had been 
properly positioned" (NTSB, 1988b, p. 21). 

6.1.2. Speed Accuracy Trade-Off 

Another psychological factor that has an effect on checklist performance is the relationship between 
the speed of performing the checklist and the quality (accuracy) of the check. Laboratory research 
showed a very definable relationship between response-time and error-rate (Wickens, 1984). 
Therefore, if the pilot scans the appropriate panel(s) rapidly because of time pressure, the accuracy 
of his perception will suffer and the probability of error will increase. 

6. 1 .3. Realization of Checklist Importance 

The relationship between the task and its expected outcome is another factor that affects checklist 
use. Without the crew witnessing its apparent effectiveness, the redundant function of the checklist 
can sometimes lead to a decline in the perception of the task's importance. This is somewhat 
analogous to the use of seat belts in a car: although most experienced drivers are aware of the 
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consequences of not wearing a seat belt, the individual's personal experience about the likelihood 
of an injury while not wearing a seat-belt is relatively low. The same applies to checklist usage. 

Summarizing, the combined effect of expectations, experience, and the pattern-analyzing 
mechanism is a double edged sword. On one side, this ability makes the user flexible and faster in 
responding to multiple conditions. On the other side, it can lead the operator to make a disastrous 
mistake just because part of the information which was collected quickly or without sufficient 
attention appeared to match the expected condition. 


6.2. CHECKLIST PHRASEOLOGY 

Although the wording used in the checklist procedure is quite rigidly defined on the checklist card 
itself, it appears that there are several problems associated with checklist phraseology that have led 
some pilots to err while conducting the checklist. 

6.2.1. Standard Phraseology 

Communication between two different entities in the aviation system is never perfect: distortion, 
high ambient noise, interruptions, hearing loss, and confusion always tend to reduce the quality of 
the physical stimulus. Operators usually compensate for this by increasing the level of expectancy: 

...Taxiing out for takeoff (runway 23L) ground control told our flight to switch to tower 
frequency and also stated "taxi short of taxiway Lima" - but due to background noise and 
cockpit workload (manifest check and checklists), clearance was interpreted as "taxi via 
taxiway Lima... (ASRS, #29080) 

To reduce the potential danger of communication expectancy, several design guidelines that are 
applicable to checklist use are described in the literature: 

1 . Restricting vocabulary size, and use of phonetically balanced words (Kryter, 1972). 

2. Increasing sequential constraints between items (Wickens, 1984). 

3. Employing frequently used words. (Wickens, 1984; Hawkins, 1987). 

6.2.2. Non-Standard Phraseolog y 

Non-standard phraseology in task-oriented communications is an integral part of many aviation 
systems. Billings and Cheaney (1981) state that "non-standard or ad-hoc procedures or 
phraseology" is one of the behavioral attributes frequently found in association with information 
transfer problems (p. 86). Most companies require standard phraseology for the checklist 
procedure; however, the authors* field studies and interviews indicated that some pilots violate 
these standards for the following reasons: 

1 . The pilot thinks the standard checklist phraseology is too cumbersome and/or not adequate. 

2. The pilot wishes to be unique. 

3 . The pilot wishes to show a high level of competency 
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The first of these points is primarily the result of improper discipline, and/or may portray 
management reluctance to obtain feedback from line pilots regarding checklist design problems. 

The second is quite common among many professional operators who frequently use radio 
communications (i.e. pilots, controllers, mariners). While required to use standardized 
communication, operators tend to presume they lose their individuality while using this 
phraseology, and the only way to restore this significance is to perform communications in a 
unique way— demonstrating a personal style, perhaps to add "humor" (e.g., referring to fuel as 
"gasoline"). Another factor is the level of knowledge of the other crew member s behavior. Several 
pilots that were interviewed commented that when checklists are constantly read to the same crew 
member, it was "tempting" to believe that he or she will comprehend any unique and non-standard 
phraseology. 

Although most checklists require short and precise communication, departure from standard ^ 
phraseology was observed in the field studies. This was noted in the initiation calls lets do it , in 
the challenge-and-responses calls, "fuel— we are OK"; and use of hand signals (thumb up) to 
indicate completion of task-checklists and items. By not employing standard phraseology, the 
following may occur: 

1 . The other crew member might not detect a checklist error. 

2 . The other crew member might not be able to follow the sequence of the checklist 
procedure. 

3 . The other crew member might confuse the checklist callout with other intra-cockpit 
communications. 

4 . The seriousness of the checklist and standardized checklist procedures is belittled in the 
eyes of other crew members, particularly if committed by the captain. 

6.2.3. Checklist Ambiguity 

It is fascinating to note how many ambiguous terms can be found in such a verbally restricted 
procedure as the checklist. The ASRS database has numerous reports where checklist responses 
were improperly called and created confusion on part of the other crew members. Many checklists 
examined by the authors employ the ambiguous responses "set," "check," "completed," etc. to 
indicate that an item is accomplished. Instead, we believe that whenever possible, the response 
should always portray the actual status or the value of the item (switches, levers, lights, fuel 
quantities, etc.). One ASRS report details the problems with this ambiguity and also provides the 
solution: 

...During taxi segment, the first officer normally sets V 2 in the autopilot for proper display 
and auto- throttle operation. Being tired, rushed, and late at night, Vref+10 (landing speed) 
was left in the window, and the Vref white bug was still left as is... .When EPR and 
AIRSPEED were called on the checklist, we looked, saw our bugs in a normal set up and I 
replied "checked and set." It wasn't until the takeoff roll we noticed the incorrect setting 
(our approach setting). I strongly feel that we need to back the response portion of our 
checklist with "what you see" responses not just "checked and set," etc. For example 
"Altimeters 30.10" and not "checked"; airspeed/EPR 125/2.00. It takes as much time to say 
it like it is as to say "checked and set." The problem is "checked" and "set" can be said too 
easily without any sound verification... (ASRS #76798) 
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The same logic applies to calling out "V speeds" while checking the airspeed bugs prior to takeoff 
and landing. In most checklists examined by the authors, the response to the "V speeds" challenge 
is "check" or "set." However, by calling out the actual numbers (VI, Vr, V2, Vief, etc.) as a 
response, the pilots have a verbal confirmation to the setting they have placed on their respective air 
speed indicators. In addition, this overt recall can aid in the mental preparation for takeoffAanding. 

6.2.4. Ambiguity of Names Given to Displavs/Controls 

Operating several types of aircraft produced by different airframe manufacturers results in a unique 
phraseology and standardization problem in display/control names. It is surprising to note how 
many different names can be assigned to the same control, and how prevalent this non-standard 
wording is in the industry. For example, the engine master switch used in starting the engine can 
be also termed master-lever, start-lever, or engine-switch. A similar wording problem is found 
with throttles, thrust-levers, power-levers, engine-power-control, etc. To reduce this variability, 
one airline has undergone a program to standardize nomenclature of documentation (checklists, 
flight manuals, training manuals, etc.) between different models from various airframe 
manufacturers in order to reduce this variability and aid pilots transitioning between aircraft types. 


6.3. USE OF PROCEDURES 

Humans, as opposed to machines, are very flexible in adapting to changing conditions. However, 
this flexibility is purchased at a cost. Human performance is variable-it fluctuates and therefore 
may produce errors. In addition, humans can sometimes "become creative in changing their 
responses while it is not optimal to do so" (Wickens, 1984, p. 7). Standardization of any 
procedure, including those involving checklists, is intended to reduce this ever-existing variance in 
human behavior. Procedures, when applied in a disciplined and standard manner, are intended to 
support the human operator by providing a firm foundation for the task, on which he or she can 
depend during a "low" in performance (NTSB, 1988a). 

One of the oldest and most pervasive problems in aviation is the failure of pilots to follow standard 
rules and procedures (Orlady, 1989). This problem was quantified by Lautman and Gallimore 
(1988) in their analysis of 93 major hull loss accidents. They reported that the primary factor in 
crew-caused accidents was "Pilot deviation from basic operational procedures" (Table I, p. 2). 

Many models of human errors and analyses of human contributions to systems failure focus on 
errors of cognition and perception, but usually neglect the social context in which errors are made. 
Reason (1988) defines these neglected factors as violations- "deviations from those practices 
deemed necessary to maintain the safe operations of a hazardous system" (p. 3). Violations can be 
categorized as intentional and unintentional. Intentional violations fall into the obvious definition of 
sabotage, while the unintentional ones fall into the much-detailed definition of human error such as 
slips and mistakes. The authors believe that intentional misuse or non-use of checklist occupies the 
middle ground: having some degree of intent, yet not involving the goal of damaging the entire 
system. 

The above may explain some of the reasons why a number of pilots consciously deviate from 
performing the checklist procedure as prescribed in the SOP. The solution for this type of problem, 
however, requires considerable attention from operational management, check airmen, flight 
standards committees, as well as from training departments. 
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6.4. COCKPIT RESOURCE MANAGEMENT 

Cockpit resource management (CRM) has been a topic of much inquiry in the last decade, with 
concern being focused on the coordination, social processes, and combined performance of a multi- 
pilot flight crew. This approach gained more support when it became apparent that human error 
plays "...a progressively more important role in accident causation as aircraft equipment has 
become more reliable" (Nagel, 1988, p. 266). Wiener (1989) defines the term "cockpit resource 
management,” as 

The manner in which individual crew members support each other, the roles played by the 
captain as pilot in command (PIC), and the role of the first officer and flight engineer. It is 
an encompassing term which includes crew coordination, communication, the use of 
human and inanimate resources of information both within and without the cockpit, role 
definition, the exercise of authority by the captain, and assertiveness by the other crew 
member(s). (p. 119) 

Checklist procedures are accomplished by coordinated actions and communication between the 
captain and other pilot(s). In addition, the checklist procedure is designed in such a way as to 
assign very distinct role definitions. It also requires assertiveness from subordinates when the 
checklist is not initiated properly by the captain, as well as firm leadership by the captain when the 
subordinate officers are the culprits. These interactions between the CRM and the process of 
checklist usage makes CRM a valid area of interest in understanding checklist problems. 

6.4. 1 . Checklist Management 

In 1977, the late H. P. Ruffell Smith conducted a full mission simulation experiment to study 
errors, vigilance, and decision making capabilities in flight crews. One of the issues addressed in 
his report was "the <captain's> failure to anticipate the overloading of a crew member by a certain 
combination of circumstances" (Ruffell Smith, 1979, p. 28). An ASRS report speaks to the same 
issue: 


The captain made the first takeoff with total disregard of the flight engineer who was 
somewhat behind in his duties due to a difficult time in contacting load control for our load 
closeout. The result was that the BEFORE TAKEOFF was finished 10 knots prior to VI. 
This was not the flight engineer’s fault, but the captain's. He could have waited 30 seconds 
for the flight engineer to properly perform his duties. End result, three guys in the cockpit 
that don't talk to each other for 2 days. No mutual support and little crew coordination... 
(ASRS, #74174) 

The initiation of any task-checklist must be carefully evaluated by the captain before making this 

call. The captain must make the following cockpit management considerations: 

1 . Are other pilot(s) overloaded with other tasks? 

2. What are the consequences of having the other pilot(s) running the checklist and, therefore, 
not participating in the current task? 

3 . What is the likely outcome of delaying the checklist because of the above considerations? 

Following the initiation of the checklist, the captain must also constantly evaluate the quality of the 

checks performed by himself and other pilot(s). If due to any factor (i.e., interruptions, distraction. 
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time limitation, workload) the quality of checklist performance appears to be below the acceptable 
level, it is the captain's responsibility to stop the checklist, allocate additional time for proper 
execution, and possibly "run” the checklist again. 

6.4.2. Sharing of Information 

There are numerous sources of information in the modem cockpit. However, not every piece of 
information is available to all pilots instantly. Obtaining information about aircraft configuration is 
even more critical in modem two-pilot cockpits. Compared to three-pilot cockpits it is physically 
difficult in advanced two-pilot cockpits for one pilot to observe what the other is doing. This is 
mainly because displays of many flight functions have migrated from the panels to cathode ray 
tubes (CRT) driven by on-board computers. In older technology airplanes, one scan on the 
appropriate panel, such as the engineer's panel, would have given the pilot the required 
configuration information. In modem aircraft, this requires some page manipulations on the 
computer's display. The result is a need for more crew coordination and more effective information 
transfer in the cockpits of modem airplanes (Wiener, 1989). 

6.4.3. Role Structure 

Several factors combine to make the role structure in the airline cockpit one of the most unique in 
any organization: the time critical operation, the catastrophic potentials, and the function of the 
leader (captain) as a hands-on operator. This has been very stringently defined and maintained in 
the last 60 years, and its associated discipline and behavior became the foundation for operation in 
this high risk industry. However, increased automation and use of computers in the flight deck 
"...tend to induce a breakdown of the traditional and clearly defined role of the pilot flying versus 
the pilot not flying, and a less clear demarcation of 'who does what' than in traditional cockpits" 
(Wiener, 1989, p. 178). 

Role structure problems were cited by the Safety Board as some of the components that led to 
omission of flap/slat setting in the Northwest Flight 255 accident. Northwest procedures required 
that during ground operations the captain is to initiate each checklist by calling for it by name; if the 
captain does not call for the checklist, the first officer is required to ask the captain if he is ready to 
run the checklist. However, except for the BEFORE START checklist, the captain of flight 255 did 
not call for the AFTER START, TAXI, and the BEFORE TAKEOFF checklist, nor did the first 
officer ask the captain if he was ready to perform any of these task-checklists before he started 
reading the items. Furthermore, checklist items that required actions and responses from the 
captain were performed by the first officer alone (NTSB, 1988a). The fundamental and well- 
defined role structure in the cockpit was broken down by the captain, who apparently yielded some 
of his specific roles to his subordinate, and by the first officer who assumed these roles. 
Concerning checklist initiation and checklist use, the first officer assumed the role of the leader, in 
addition to his defined duties. Consequently, the first officer was overloaded by a combination of 
his own duties and some of the captain's duties. But at this point there was nobody to back up and 
verify his own actions. This condition "rendered the crew more susceptible to distractions and 
memory lapses" (NTSB, 1988a, p. 57). 

A similar, yet reversed, breakdown of crew role structure and coordination was cited by the Safety 
Board in the Air New Orleans BAe J-31 accident. In this accident, the crew failed to comply with 
the BEFORE TAKEOFF checklist by not properly advancing the RPM levers to the high RPM 
position (see section 6.1.1). The captain (PF of that leg) stated that "...he had personally advanced 
the RPM levers rather than the first officer, even though the company procedures required the non- 
flying pilot to advance the RPM levers" (NTSB 1988b). 
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Just as the first officer of Northwest Flight 255 assumed the captain's duties, the captain of the Air 
New Orleans BAe J-31 overloaded himself with his copilot duties. The BAe J-3 1 captain used his 
authority to "short-cut" a structured procedure, broke an element of crew coordination, and may 
have somewhat "pushed" his first officer "out of the loop." 

6.4.4. Mutual Supervision 

Supervision of the other crew member while performing the checklist and other tasks becomes 
somewhat difficult in modem aircraft. Wiener (1989) posed this question to a sample of 166 B-757 
pilots. Figure 9 shows that the captains leaned toward the view that supervision of first officers is 
not easier in the B-757. Many first officers chose the neutral opinion (probably because the 
question related to captains and not to first officers); the reminder largely leaning toward the notion 
that supervision is not easier in the automated environment. 

Another problem in mutual supervision within the cockpit is over-cohesiveness of the crew. 
Although it might seem somewhat contradictory to many examples of CRM incidents in which the 
captain was over-governing the rest of the crew, the opposite extreme can also create problems. 
The crew, by being too cohesive, may produce errors because of several reasons. First, the 
individual may fear coming into conflict with other(s) who have decided to deviate from a 
procedure. Second, the individual may be overly confident in his fellow crew members, based on 
satisfactory past performance: 

First Officer forgot to read flap setting on pre-takeoff checklist and made takeoff with flaps 
up. ...First officer said he was distracted by tower call. The captain says they were behind 
schedule...and with this highly competent first officer <italics added>, he had become a 
little complacent ... (ASRS #58147) 

6.4.5. The Captain as the Pilot Not Flying 

Most airline pilots obey the tradition that during a flight (several legs) without foreseeable adverse 
conditions (such as weather or a difficult departure/approach) the captain and the first officer 
alternate, on each leg, the Pilot Flying (PF) and Pilot Not Flying (PNF) duties. Having the first 
officer as the pilot flying does not relieve the captain of his duties as pilot in command; but rather it 
adds a supervisory task to his usual role. 

From the authors’ field studies, it was evident that some of these problems are also apparent in 
checklist use. Out of six different crews that the first author observed flying the B-757, two crews 
did not perform the flight phase checklists during one leg of their flight. This was clearly observed 
as the checklist card did not leave the glare-shield slot throughout part of the ground and all the 
airborne phase (Taxi to Landing), nor was the yoke-mounted checklist ever used (it was covered 
with the approach plate). There were several common factors to these occurrences: 

1 . The captain was the pilot not flying (PNF). 

2. The leg was very short and followed a long leg. 

3 . The type of plane was a B-757. 

The captains ignored the checklist procedure. Yet, the first officers never made any comment 
regarding this clear deviation from standard procedure. The authors believe that the short length of 
the leg (compared to the previous long leg) and the fact that the flights were the final ones for the 
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In the B-757 it is easier for the captain to supervise the first officer 

than in other planes 



agree disagree 

Pilot's response 

Figure 9. From Wiener, 1989, Figure 36A, p.122 
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day also played a part in this case. In addition, the minimal length (number of items) of the B-757 
checklist used by this company made it also inviting to commit this checklist to memory. 

Checklist performance is affected by the way individuals perform as a crew. Poor crew 
coordination and diminished role structures can lead to omissions and mistakes. And when these 
omissions interact with component failure, the result may be an incident or an accident 
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7. ANALYSIS AND DESIGN ISSUES 

The following discussion will detail the considerations in designing a checklist. The use of a paper- 
type checklist and challenge-response method are assumed. 


7.1. TASK ANALYSIS 

In the previous sections we determined that the use of checklists by FAR Part 121 and 135 
operators is a task in itself and not merely a memory enhancement tool. Therefore it seems logical 
to analyze this task by using task analysis, a formal human factors method. Drury, Paramore, Van 
Cott, Grey, and Corlett (1987) define task analysis as a method ’’which describes and analyzes the 
performance demands made on the human element of a system. By concentrating on the human 
element in the system, it can compare these task demands with the known human capabilities' 

(p. 371). 

In general, the process of analyzing a human function in a system is based on three different, yet 
interrelated analyses. The first is the hardware/software operating process which is the foundation 
for the entire task analysis. The second stage is the task classification and description which details 
the human task requirements and provides the information needed to perform the work. The third 
is the actual analysis, interpretation, evaluation, and transformation of the task demands based on 
the knowledge of human capabilities (Drury et al., 1987). 

7.1.1. Defining the task-checklist 

The entire flight checklist is made up of several task-checklists that follow the sequence of flight 
(PREFLIGHT through SECURING THE PLANE). There are many ways one can divide and 
categorize the flight into separate phases, and hence task-checklists. For e xamp le, the ground 
phase can be divided to several checklists: BEFORE ENGINE START, AFTER ENGINE 
START, BEFORE TAXI, TAXI, BEFORE TAKEOFF. Another approach is to divide the same 
segment into only two task-checklists: BEFORE ENGINE START, and BEFORE TAKE OFF. 

Checklists of older technology airplanes such as the L-1011, DC-9 series, and DC-10 usually have 
a long list of items on the BEFORE ENGINE START. Some aircraft have as many as 76 items on 
the checklist of the first flight of the day, and 37 items to be checked at intermediate stations. Many 
pilots have complained about inadvertently skipping items in long checklists. Not surprisingly, 
Swain and Guttman (1983) in their study of nuclear power plant operations "recognized the fact 
that as the list of items grows, there may be a higher probability of overlooking any given item" 
(chap. 15, p. 13). The above does not necessarily imply that every task-checklist should be divided 
into smaller lists. However, having one task-checklist with 76 items may be somewhat absurd. 

The limited capacity of short-term memory (STM) is one of the most severe constraints on human 
performance (Card, Moran, and Newell, 1983; Sanders and McCormick, 1987). Yet many tasks, 
such as long checklists, place unrealistic demands on this memory (Swain and Guttman, 1983). 
Nevertheless, certain techniques that aid human abilities under these limitations are documented in 
the literature. Miller (1956) has formulated the "seven plus-or-minus two" rule, quantifying the 
normal range of items that can be stored in the working memory. He further recognized that people 
can chunk (cluster) information into defined units regardless of the length or size of the unit. The 
chunk is created when two or more items share a common factor that aids in "gluing" these items 
together. 
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Wickens (1987) states that ’’...where printed information is read, stored and used (as in 
instructions, procedures, etc.), the retention process can be aided by <1> determining the logical 
chunks that are grouped together in memory, and <2> by physically separating these chunks from 
others” (p. 82). Using these principles, chunks can be employed in designing the structure of a 
long task-checklist by: 

1 . Grouping the items corresponding to a system such as pressurization, hydraulic, electrical, 
etc., to chunks of checklist items. 

2 . Physically (graphically), separating these chunks while designing the layout of the checklist 
card. 

If the checklist designer will employ these principles in checklist design and layout, it will be easier 
for the pilot to index and follow the order of the items while conducting the checklist, as the 
sequence of movement will be within and between chunks. 


7 2 THE ORDER OF CHECKLIST ITEMS. 

In a paper checklist, the order of the items is the only indicator as to the operator s point of 
progress in the checklist ("where are we on the checklist...? ). Therefore, the order of the checklist 
items is an important structural format in an effort to reduce the potential for failure while 
conducting this procedure. When designing the order of the items in a task-checklist, the following 
factors should be considered. 

7.2. 1 . Systems Operat ional Sequence 

When operating a complex system like an airplane, it is clear that operations must be sequenced 
according to the activation and operation of units and systems. For example, it would not be 
appropriate to check hydraulic pressure prior to activation of hydraulic pumps. This type of 
sequencing is most stringent in starting the engines, and in activating related systems such as 
electrical, hydraulics, air conditioning, etc. Other duties such as checking altimeters, setting speed 
bugs, speed brakes, lights, no smoking signs etc., are not so stringently coupled with prior 
activities, and in such cases, the designer has flexibility to allocate these items in a sequence that 
will be most advantageous for his structured design. 

7.2.2. Patterns of Motor and Eve Movements 

In the cockpit of an airplane, the instruments, units and system panels are arranged in a certain 
"geographical” locations according to frequency-of-use, criticality, and other human factors 
considerations. In order to facilitate a logical flow while initially configuring the plane, training 
departments require that this task will be conducted in a particular sequence of motor and eye 
„ movement called the "flow-pattern." 

Exactly the same spatial technique can be adopted for the sequence of verifying checklist items. 
Additional enhancement of the procedure can be achieved if the sequence of accomplishing the 
items and chunks follows a logical and consistent order. For example, one can conduct the 
BEFORE ENGINE START checklist from the aft (upper) portion of the overhead panel, moving 
with the checks toward the lower part of this panel. By using a top-to-bottom order of checking 
panels and items, the design can accommodate population stereotype of order and sequence (top-to- 
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bottom is a common arrangement), as well as some biomechanical considerations (it is less 
fatiguing to move the arms and the head from above to below than vice-versa). 

The use of appropriate flow patterns in conducting the checklist procedure can aid the process in 
the following ways: 

1 . Standardization of the checklist flow among pilots. 

2 . Making the checklist sequence run parallel to the initial set up flow-patterns (which are 
done before ”running ,, the checklist), and thereby simplifying the learning process and the 
daily use of the checklist process. 

3 . Making the checklist actions logical and consistent (as opposed to intermittent) in the motor 
movement of the head, arms and hands. 

There are many advantages for using the verbal-print (challenge-response) and the spatial (flow- 
patterns) perceptual channels while conducting a step-by-step procedure such as the checklist. 
Booher (1975) used several combinations of spatial (pictorial) and print formats for procedural 
instruction in operating a control panel. He reported that action-response type information is more 
efficiently presented in print instructions, while spatial presentation is better for organization of 
perceptual-motor actions. 

Booher suggests that only one unique combination of the two processing channels will yield better 
accuracy from the operator— ' when the pictorial mode is used to aid in selection and organization of 
a range of perceptual-motor actions and the verbal material is available to confirm specific tasks 
within the range" (p. 276). 

Therefore, by using a combination of spatial flow-patterns and verbal confirmation, the designer 
can maximize the effectiveness of the checklist procedure. The flow pattern aids in sequencing the 
checks between cockpit panels, while the printed checklist confirms the individual items within the 
panel. The authors also believe that multiple channel redundancy can aid the checklist procedure in 
instances where this highly sequential procedure is interrupted. The use of spatial organization will 
provide an additional pointer as to the location of the interruption on the geographical layout of the 
cockpit. 

In addition to visual verification of the check item, motor movement such as touching controls and 
displays ("muscle memory" as some name it) is also an effective enhancement for the verification 
process. The use of the hand to guide the eye while using the flow pattern can substantially aid the 
checklist procedure by combining the mental sequencing process with motor movements. 
Furthermore, the use of the hand and finger to direct the eye to an alphanumeric display or control 
can aid in fixating the eyes on the specific item and prevent the eyes from wandering away from 
that indicator. 

7.2.3. The Operational Lode 

Certain tasks that are part of the checklist are dependent on internal and external agents such as 
flight attendants, gate agents, refueling agents, etc. When considering the chronological and logical 
sequence of the checklist, the influence of these uncontrollable entities must be considered. For 
example, it would be inappropriate to require pilots to check the closure of cargo doors during the 
PREFLIGHT checklist, when, due to the way the entire system operates, the doors are closed just 
before engine start. 
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A task-checklist that includes items that do not run parallel to the activities occurring around the 
plane has an inherent disadvantage. Omission of checklist items sometimes occurs when an item 
that could not be completed in sequence (because of the above limitations) is deferred by the crew 
to be accomplished later on. But since the traditional paper checklist has no means of prompting the 
pilot about such unaccomplished items, the deferred item is stored in the pilot s short-term 
memory. However, due to the limitations of this memory, coupled with time constraints, and the 
vulnerability of the crew to distracting events, the likelihood of this item being omitted is relatively 
high. 

The following narrative from a report submitted to the ASRS illustrates how the crew deferred 
checking the fuel on the preflight checklist and what were the consequences. 

Prior to departure from Denver, as the preflight checklists were being accomplished, it was 
noted that the plane was not fueled yet. The crew continued <deferred the item for later 
completion> in accomplishing the rest of the checklist and related preflight duties. 
Approximately ten minutes after takeoff the second officer noted that the plane was not 
fueled. The flight returned to Denver for additional fuel. At company Denver facilities, 
experience dictates that dispatch fuel is not on board prior to completion of the pre-flight 
checklist in approximately 75% of departures... (ASRS #2855) 

The authors take the position that because each company has its own operational logic, this should 
have a visible effect on the sequence and method in which the checklist is conducted. Such unique 
operational factors make it unlikely that one could design a "nationwide" normal checklist. 

7.2.4. Sequencing of Very Critical Items 

In section 6.4 we stated that one of the important duties of the PIC or PF in checklist management 
is to call for the task-checklist at the appropriate time. Often, the pilot manipulating the controls will 
call for the task-checklist when the workload is low and the probability of interruption is also low. 
For example, the captain will usually call for the TAXI checklist after the plane is clear of all 
obstacles on the ramp, all systems are working, instructions for taxiing are known, etc. At this 
instant, the probability of successfully accomplishing the first item on the TAXI checklist is the 
highest. However, the probability of accomplishing the subsequent items slowly diminishes as 
time progresses, since there is more chance for interruptions and distractions to occur, all 
interfering with the checklist sequence. The authors' position is that very critical items should be 
completed first on the task-checklist, and not last. 

7.2.5. Duplication of Checklist Items 

Several carriers have opted to repeat a number of checklist items for redundancy and therefore 
reduce the probability of skipping an important item by the flight crew. Ironically, this is very 
common in checklist philosophies that employ detailed and long lists of items. Although this 
additional redundancy in the checklist might prevent an item from being missed, overemphasis of 
items might diminish the crew's overall checklist performance. 

Conversely, duplication of a very few highly critical items ("killer items") that are based on 
possibly transient data, can be beneficial. For example, items such as flap/slat setting are calculated 
according to several variables such as gross weight, runway length, etc. Flight crews are usually 
briefed prior to taxi to expect a certain runway, and calculate the above settings accordingly. 
However, due to weather changes or takeoffs from a taxiway intersection (to avoid a long line), 
there might be a need to recalculate and therefore set these items again prior to takeoff. 
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Regarding Northwest Flight 255, the Safety Board could not determine conclusively why the first 
officer did not extend the flaps/slats prior to takeoff. One of the Board's speculations was that 
while "anticipating a different flap setting due to the runway change, the first officer might have 
elected to delay the deployment of the flaps until a specific runway would be assigned" (NTSB, 
1988a). 

To prevent such delay of action that might lead to omission, the authors feel that perhaps one or 
two very critical items (i.e., flaps/slats, trim), should be set and checked in one of the ground 
phase task-checklist, and then checked again before takeoff. This should provide an additional 
safeguard against configuration omissions or errors. 
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8. THE CHECKLIST AS A SYSTEM 

The manner in which checklists are designed, taught and used can be examined by employing 
several concepts that are used in systems analysis. This is the topic of this section. In addition, m 
this section we will also discuss the unique contributions of human error to incidents and accidents 
in high risk systems. 


8.1. SYSTEMS 

Systems are found everywhere we look. They range from agencies, firms, and universities to more 
complex systems such as nuclear plants, chemical plants, aircraft, etc. Systems, as any human 
made structure, are found to fail occasionally. However, when nuclear plants and aircraft fail, the 
results can become catastrophic. Therefore, these systems can be defined as high risk systems. 

Leading to any system failure is a component (unit/part) failure. Yet, one of the main reasons for a 
system failure in high risk industries is the unpredictable interaction of several failed components. 
Each one by itself is not critical enough to create a total system failure, but the timely combination 
of these singular component failures may lead to the breakdown of the entire system. An example 
of such a combination is the Northwest Flight 255 accident. The singular misuse of the ground 
phase checklists, the failure of the CAWS system, or the breakdown in crew coordination would 
probably have not led to an accident. But the interaction between these component failures striped 
the system of its redundant defenses and led to a system accident. 

Perrow (1984, 1986) argues that since designers expect everything to be subject to failure, they 
guard against each singular failure with one or more safety devices. What system designers cannot 
anticipate is that "...multiple failures will interact so as to defeat, bypass, or disable the safety 
devices" (Perrow, 1984, p. 116). These accidents are rare yet normal for the systems that host 
them. Therefore, in a sense, they are normal accidents (Perrow, 1984). 

8.1.1. Systems Characteristics 

Perrow (1984, 1986) analyzes system potential for accidents according to two characteristics: the 
type of interactions between system components, and the level of dependency (coupling) between 
components within the system: 

Type of interactions. As systems grow in size and diversity, they usually become more 
complex. Systems experience more and more interactions that were not formulated by the system 
designers. These unexpected interactions may manifest themselves in incidents, and from time to 
time in a system accident. In contrast, other systems, such as most assembly-line production 
facilities, incorporate more linear and/or simpler interactions, and therefore a potential interaction 
can be more obvious, foreseen, easily understood, and thereby contained. 

Dependency (coupling! . This term relates to the amount of slack or buffering between 
system components, and usually this factor is time dependent. In a loosely coupled system , such 
as post office or motor vehicles agency, delays are possible, and the outcome of the system (the 
product) will not change while waiting. In a tightly coupled system , such as dams or chemical 
processing plants, there is only a small slack or buffer between system components. Time is a 
critical factor: the product cannot stand by or wait until attended to. Instead it must be processed 
immediately and sequentially. 
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8.2. THE CHECKLIST 


The same characteristics of a large system apply for a smaller system such as the checklist. Based 
on the first concept, the checklist is a linear stem. It requires verifying configuration items in a 
linear and independent manner. 

As for the second concept, coupling, one may argue that the checklist is a loosely coupled system. 
The checklist process can be stopped or delayed, and there is almost no critical time dependency 
between items. However, in daily line operations the reality is sometimes very different, and the 
checklist can easily be transformed into a non-linear and tightly-coupled system. This 
transformation can be caused by (1) operators, (2) designers, and (3) management. 

g.2,j, Op er a t ors 

The cockpit crew, in particular the PIC, can tightly couple the checklist procedure to other tasks 
such as starting engines, takeoffs, landing, etc. This point is amplified in section 6.4. When a 
checklist is tightly coupled, the buffers embedded in the system (redundancies and backups) are 
bypassed, and the ability of the crew to recover from a failure is diminished. 

8.2.2. Checklist Desig n 

The same concept of tightly coupling the checklist to other systems is sometimes carelessly 
"designed into" the checklist procedure. Several companies require that a TAKEOFF checklist be 
accomplished on the active runway , or just prior to entry onto the runway. In this case, the 
TAKEOFF checklist is tightly coupled with other takeoff tasks such as ATC instructions, other 
planes on final approach, and with the pilots’ mental preparation for takeoff ("V" speeds, wind, 
noise abatement procedures, etc.). 

In several other checklists examined by the authors, critical items such as flaps/slats, stabilizer 
trim, etc., were placed at the end of the BEFORE TAKEOFF and TAXI checklists. For example, 
in the B-727 checklist that was used during the Delta Air Lines Flight 1 141 accident, the critical 
item "Flaps" is listed in the small portion of the TAXI checklist that follows the DELAYED 
ENGINE START checklist (Figure 10). The Board stated that 

The flight crew began to start the No. 3 engine when they believed they were No. 4 for 
takeoff. Within 15 seconds they received the clearance from the tower controller to taxi 
onto runway 18L and hold for takeoff. At this point, the CVR shows a distinct difference 
in the crew’s conduct in the accomplishment of the checklist. Apparently, the second and 
first officer recognized the need for expeditious completion of the remaining checklist to 
prevent delay on the runway. (NTSB, 1989, p. 61) 

Listing critical items at the end of the TAXI or BEFORE TAKEOFF checklists is probably done 
for the sake of checking these items closer to the segment in which they are to be employed. 
However, by using this rational, the designer provides the potential for tight coupling of these 
critical checks with other sub-tasks. The authors' position is that these critical checks should be 
completed earlier in the ground phase in order to decouple the critical items from the takeoff 
segment as well as to allow enough time (buffers) for the crew to detect and recover from a 
configuration failure. 
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■ INTERMEDIATE STOP ITEMS 
Jk. S/OS FUNCTION 
NO CHALLENGE 


B-727 PILOT’S CHECKLIST ^ 
(- 232 & -247 Modified) 


FIRST FUQHT Of MY CHECK THE FOLLOWING 


AFTER START 


FAA APPROVED 
DATE 3-15-80 


T STANDBY RUDDER FLIGHT DIRECTORS . 

, ANTI-SKID STAB TRIM • 

* ANTI-ICE AUTOPILOT ! 

' PITOT HEAT | 

~ BEFORE START ’ 

■ A/C STATUS 4 LOGBOOK CK 

■ FUEL REOD ON BOARD 

OXY MASK/REG/INTERPHONE CK 

STANDBY RUDDER • OFF 

FLIGHT CONTROL SWITCHES ON 

ANTI-SKID OFF 

STALL WARNING CK 

INSTRUMENT COMPARATOR CK 

EMERGENCY EXIT LIGHTS ARMED 

«2 ENG ACCESS DOOR LIGHT OFF 

ENGINE START SWITCHES OFF 

■ SEAT BELT & NO SMOKING ON 

■ WINDOW HEAT ON 

ANTI-ICE CLOSED 

PITOT HEAT OFF 

INTERIOR & EXTERIOR LIGHTS SET 

■ NAVIGATION LIGHTS [247] ON 

■ ENGINE FIRE WARNING SYSTEMS CK 

ALT & FLT INSTS SET/CROSSCK 

COMPASS SELECTORS SLAVED 

GPWS CK 

STATIC SOURCE SELECTORS NORMAL 

MACH AIRSPEED WARNING CK 

INDICATOR LIGHTS & APD CK 

ENGINE INSTRUMENTS CK 

LANDING GEAR DOWN, IN, 3 GREEN 

■ RADAR 4 TRANS CK A SET 

FLIGHT DIRECTORS STBY 

SPEED BRAKE DETENT 

REV. THROTTLES & START LEVERS 

DOWN. CLOSED & CUTOFF 

FLAPS UP 

STAB TRIM NORMAL 

AUTOPILOT OFF 

■ RADIOS & NAV INSTS CK A SET 

■ RUDDER 4 AILERON TRIM ZERO 

STANDBY POWER CK 

■ CABIN 4 S O PREFLIGHT COMPLETE 

■ a AUTOPILOT TEST SW NORMAL 

■ a. CIRCUIT BREAKERS CK 

■ DEPARTURE BRIEFING COMP 


BEFORE DISPATCH 


A ELEC SYSTEM 

A EXTERNAL POWER A AIR 

A. GALLEY POWER 

A FUEL SYSTEM 

A SYS A A B HYD PUMPS . 

a eng 2/APU BLEEDS 

A. packs 

COCKPIT DOOR 

ENGINE INSTRUMENTS . 

ENGINE ANTI-ICE 


CK A SET 

REMOVE 

ON 

SET 

CK A ON 

AS REOD 

AS REOD 

LOCKED (PRIOR TO T/O) 

CK 

AS REOD 


TAXI 


T/O DATA COMPUTED RWY 

AUTO PACK TRIP LT 

PITOT HEAT 

AIRSPD A EPR BUGS 

AIRSPD WARN SWS 

ALT A FLT INSTS 

STAB TRIM 


■USING RWY 

REOD OR NOT REOD 

ON 

SET/CROSSCK 

SET, 3^^— (A On B) 

SET/CROSSCK 

UNITS 


When Delayed Start is Desired 


DELAYED ENGINE START 


▲ PACKS BOTH OFF 

A. GALLEY POWER OFF 

A. FUEL SYSTEM SET 

PNEUMATIC PRESSURE PSI 


DELAYED AFTER START 


ELEC SYSTEM 

GALLEY POWER 

FUEL SYSTEM 

SYS A HYD PUMPS . . 
ENG 2 APU BLEEDS . . 

PACKS 

ENGINE INSTRUMENTS 
ENGINE ANTI-ICE . . . 


CK A SET 

ON 

SET FOR TAKEOFF 

CK A ON 

CLOSE 

ON 

CK 

AS REOD 


TAXI (CONTINUED) 


a FUEL HEAT AS REOD 

a FLT GRD SWITCH FLT 

a F/O ALT VIBRATOR C/B [232] IN 

SHOULDER HARNESS ON 

FLAPS GREEN LIGHT 

FLIGHT CONTROLS CK 

NAV INSTRUMENTS SET 


BEFORE TAKEOFF 


a. *2 SYS B HYD PUMP ON 

a. APU 4 BATT CHARGER START A CK 

a. PACK(S) AS REOD 

ENGINE START 

a DOOR WARNING LIGHTS CK 

a. PACKS BOTH OFF 

A GALLEY POWER OFF 

a FUEL SYSTEM SET 

BEACON ON 

PARKING BRAKE AS REQD 

PNEUMATIC PRESSURE p SI 


TAKEOFF BRIEFING COMP 

FLIGHT ATTENDANTS NOTIFIED ACKD 

ANTI-SKID ON 

CONTINUOUS IGNITION [2321 ON 

START SWITCHES [247] FLT START 

NAV LIGHTS [232]/STROBE[247] ON 

TRANSPONDER ON 

A APU MASTER SWITCH OFF 

^ FUEL HEAT OFF 

^ APU LIGHT OFF 

A. AUTO PACK TRIP SWITCH NORMAL 

A. CSD OIL COOLER [247] GROUND OFF 


Figure 10. From NTSB, 1989, Appendix D, p. 132 


Similar tight coupling sometimes occurs while running the checklist during the taxiing segment. 
The ASRS database contains numerous incidents where conducting the TAXI checklist interfered 
with the actual taxiing task as well as with the ability of the first officer to backup the captain: 

Taxiing for takeoff while performing the pre-takeoff checklist. We were on the 
perpendicular taxiway approaching the parallel taxiway and the active Runway 8R. Both 
the copilot and I were busy checking items on the checklist.... I thought we were 
approaching the parallel taxiway when in fact we were nearing the active runway. I looked 
to my right, preparing to start a turn into what I thought was the taxiway and instead saw 
an aircraft turn on its lights and start its takeoff roll. I slammed on the brakes and stopped 
just short of the active runway. Some moments later, the aircraft roared past our nose on its 
takeoff roll....Even though we were following SOP's, I think it's bad practice to be doing 
a checklist while taxiing... (ASRS #60045) 

Taxiways are areas of tightly coupled operations, becoming even more coupled as the aircraft 
moves closer to the active runway(s). The authors take the position that TAXI checklists should be 
completed as close as possible to the gate and as far away as possible from the active runways and 
adjacent taxiways. 

Identifying and safeguarding unique configuration items is another measure the checklist designer 
can take in order to eliminate tight coupling of the checklist. Consider for example, zero flap/slat 
takeoff accidents. This lift device functions at a very crucial period— rotation and initial climb. Yet, 
during the ground phase there is no substantial control feedback and, in most cases, no out-of- 
cockpit visual cue that the flaps are set properly. Furthermore, the lack of flaps/slats has no effect 
on the pilots' ability to taxi the aircraft on the ground. 

Conversely, in some checklists, items that are not very critical are made so, consuming valuable 
time, adding workload and shifting attention from very critical checklist items to non-critical ones. 
For example, one company states in its SOP that the landing checklist should be called "complete" 
only after the flaps are deployed to their full down configuration for that approach. In some cases 
the LANDING check is completed as low as 500 feet above DH, thereby creating potential for 
distraction for the PF and PNF in this already high workload segment. While this procedure works 
quite satisfactorily in a three pilot crew where the flight engineer can be part of the checklist 
routine, this procedure apparently does not fit well in a two pilot crew. 

...We were instructed to contact tower at <the> marker on an ILS approach. As we went 
by the marker, the frequency change was initiated but not completed due to gear being 
lowered, flaps being lowered to 30 and then checklist called for with both pilots answering. 
Upon turning off <the runway>, discovered the wrong frequency....Tower advised us to 
cross runway 8L and that we had been "cleared to land." Forgot exact ceiling but when we 
were answering checklist for critical items, it was at a critical time... (ASRS #113158) 

In the approach and landing segment of the flight, the deployment of flaps/slats is used in sequence 
with other tasks if the plane is to slow down, descend and land (as opposed to the non-sequential 
action of deploying flaps prior to takeoff). This task also gives some control feedback to the pilot 
(speed, pitch attitude, descent rate, etc.). A search of the ASRS database from 1983-1989 (air 
carriers and military transports) did not reveal even one report of a crew landing without extending 
the flaps (ASRS, 1989b). However, this search revealed many reports in which crews missed 
other duties while conducting the LANDING checklist just prior to touchdown. 
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One company takes a somewhat different approach by specifying when the LANDING checklist 
should be completed and the checklist card set aside. This company requires that as soon as the 
first stage of the leading edge flaps is deployed the LANDING check is called complete even if 
more flaps are required for the full flap-down configuration. By doing this the designer eliminated 
one variable from a complex set of interactions, and made the system more loosely coupled. 


ft. 2A Management 

One of the characteristics of any tighdy coupled system is that it is efficient. Such systems are 
efficient in energy (fuel), and schedules are tightly governed in order to utilize the system to its 
maximum. However, this efficiency is purchased at a cost: a tightly coupled and extremely efficient 
system is more prone to failure (Perrow, 1986). 

The Airline Deregulation Act of 1978 has forced airline companies to confront the issue of 
efficiency as never before, or face the possibility of not surviving in a competitive environment. 
There are two factors in this context that affect checklist usage: fuel conservation, and production 
pressure ("making schedules") (Monan and Cheaney, 1990). 

Fuel conservation. Jet fuel is a major cost for the airlines; in 1989 the price increased 
about 30% in the U.S. Hence, much is done in order to conserve fuel during all segments of flight 
(taxi, climb, cruise, approach/descent taxi). One way of conserving fuel is taxiing with one or 
more engines shut down. Starting one or two engines on the taxiway adds workload to the taxi 
segment. In most cases, engine start and related checklists are coordinated and planned in advance 
by the flight crew while designating enough time for checklists and allowing enough time for 
recovering from errors. However, in some reported cases, captains delayed starting engine(s) to 
the last possible minute, only to find that engine starts, checklist procedures, ATC instructions, 
and the actual takeoff tasks were tightly coupled with one another. 

During the field study, one captain repeatedly delayed engine start even in situations where there 
was ample fuel on board, short taxi and almost no waiting line on the taxiway. The prime objective 
was to save money. However, starting the engine on the taxiway while taxiing toward the runway 
led to several problems: the quality of monitoring engine start (checking the gauges for proper N's, 
temperatures, fuel flow, etc.) was below standard; the captain was preoccupied with steering the 
plane, looking for landing traffic, and at the same time starting the engine. Being overloaded with 
the all the above he had to instruct the first officer to advance the start lever to idle detent. Here the 
captain made the task of engine start non-linear and tightly coupled with the taxiing phase, with 
almost no significant economic gain. 

Production pressures. Management pressure for "on time performance" is another factor 
that yields high operating efficiency. Air transports fly in and out of hubs with fast turnarounds. 
The Department of Transportation (DOT) monitors flight schedules in order to publish the highest 
and lowest ranking airlines in "on time performance" and thereby place another public relations 
burden on management. On-time performance pressures, or production pressures, propagate into 
the cockpit, and consequently affect cockpit management: 

On a very short turnaround, and behind schedule departure, the captain rushed the crew, 
and I (F/E) missed several checklist items and inadvertently took off with the APU running 
and several generators not on the bus-this was abnormal but not unsafe. More emphasis 
needs to be made not to rush. Especially before takeoff... (ASRS #48505) 
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The example given in the introduction of this report regarding the ferry ship Herald of Free 
Enterprise is indicative of similar production pressures and their potentially disastrous results in the 
marine industry. Because of delays at Dover (England), there was great pressure on the vessel s 
crews to sail early. A memo from an operations manager at Zeebrugge stated, "...put pressure on 
the first officer if you don’t think he is moving fast enough.. ..Let's put the record straight, sailing 
late out of Zeebrugge isn't on. It's 15 minutes early for us" (Department of Transport, 1987, p. 

1 1). These pressures for rapid turnarounds have led to several prior incidents where company 
ships sailed to sea with their bow or stem doors open. 

The checklist, as a back-up procedure which is initiated and conducted by the flight crew, is highly 
susceptible to production pressures. These pressures lay the foundation for errors by encouraging 
substandard performance (the Speed Accuracy Trade-Off effect) when the crew is rushing to 
complete the checklist. Furthermore, under production pressures, checklists are sometimes 
"...relegated to second place status in order to save time" (Majikas, 1989), leading some pilots to 
shortcut a part or the entire procedure. 

Production pressures in aviation date back to the era of the mail planes. They are part of any profit- 
making system and it is impractical to even contemplate eliminating them. Nonetheless, when these 
pressures penetrate into the zone designated for safety measures and the buffers required for a 
potential recovery from failure, the risks may exceed the benefits. 

It is certainly the responsibility of the pilots, and in particular the PIC, to preserve these safety 
buffers. However, in this business-oriented system, it is also the responsibility of flight 
management to assure that the pilot will be able to make any decision related to flight safety without 
having to consider the profit-making consequences. 

To conclude, flying is a tightly coupled and complex system by nature. Nevertheless, some 
subsystems can be designed to reduce the probability of failure. Redundancy, in particular 
engineering safety devices, are not always the ultimate solution, because complex interactions tend 
to defeat and bypass these devices. When subsystems such as checklists are made tightly coupled 
by operators, designers, and management, then at one time or another this may result in a checklist 
accident— a normal accident. 


8.3. SYSTEM PATHOGENS 

The distinction between active and latent failures can be useful in understanding checklist-related 
accidents. The distinction is based on the time required for human failure to manifest itself within 
the system. Active failures are those immediate failures created by the "first line" operators (wrong 
flap/slat setting, forgetting to close cargo doors, forgetting to fuel, etc.). Latent failures are ones 
"whose adverse consequences may lie dormant within the system for a long time, only to 
becoming evident when they combine with other factors to breach the system's defenses. Their 
defining feature is that they were present within the socio-technical system well before the onset of 
an accident sequence" (Reason, 1988). 

8,3.1 . Latent Failures in High R isk Systems 

Analyses of many accidents shows that although latent failures have provided the "spark" for 
interactions that led to accidents in high risk systems, many accident investigators focus primarily 
upon active errors and/or hardware failures, neglecting the latent ones. The history of high risk 
systems is filled with just such interactions of active and latent failures that have created 
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catastrophes. For example: the maintenance problems and improper human factors design in the 
Three Mile Island nuclear power plant control room; the defective design of the Challenger's solid 
booster rocket "0"-ring; the improper training and safety measures in the Union Carbide 
Corporation chemical plant in Bhopal, India; and many more. 

The aviation industry also provides such examples. In 1979, a New Zealand Airways DC- 10 
struck the slopes of Mount Erebus in Antarctica. The initial inquiry blamed one of the 257 victims— 
the captain . The cause: pilot error. A later investigation, prompted by the pilot union, revealed that 
incorrect data was entered into a ground-based computer which supplied the flight-plan for the 
inertial navigation system (INS). Once the flight plan was delivered to the crew there was no 
opportunity for human intervention. 

Reason (1988) borrows a term from medicine, referring to latent errors as "resident pathogens" 
because they reside within a system in the same way biological pathogens reside within a living 
body, only to manifest themselves as a result of unique set of unexpected conditions. 

The limitations and the deficiency of the traditional paper checklist in preventing human error, the 
inadequacy of several checklist concepts in the industry; the interaction between social issues and 
checklist performance; and the numerous checklist accident/incident reports listed in the NTSB, 
ICAO, and ASRS databases, lead the authors to believe that the traditional flight-deck checklist 
contains resident pathogens. These pathogens lie within the system in which the crew and aircraft 
operate, only to manifest themselves with a unique interaction of human failure and machine 
failures. Engineering defenses, such as configuration, warning, and alerting systems do not offer 
absolute protection against these pathogens. Furthermore, the social issues that surround checklist 
usage mice these pathogens even more difficult to control. 
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9. CONCLUSIONS 


We have discussed throughout this report the design weaknesses of the traditional (paper) checklist 
device and the limitations of the humans who interact with it. Other aviation systems also closely 
interact with checklist use. These interactions, if not properly accounted for in the checklist design 
process, may combine to reduce the effectiveness of this procedure. 

Nevertheless, we strongly believe that merely improving the engineering design and the procedural 
sequence of die checklist will not eliminate the problem. The pilot is still the center of this task, and 
the socio-technical environment in which he operates has a substantial effect on checklist 
performance, regardless of the type or method in use. Since the pilot is in control and will continue 
to be so in the foreseeable future, accommodating the human strengths and limitations in 
conducting this procedure should be at the heart of any checklist design. In short, checklists must 
be "human-centered.*' It must be clearly understood by all parties involved in checklist design that 
if the individual captain chooses not to use the checklist for any reason, no one can force him to 
use it. 

The question the aviation industry and governmental regulators must now confront is whether the 
current checklists are in themselves resident pathogens. The authors conclude that the traditional 
flight-deck checklist design and some of the checklist concepts in the industry do contain resident 
pathogens. Evidently, the extraction of these pathogens is difficult. The authors, however, hope 
that this report and the guidelines listed in Appendix A will aid those individuals who are 
responsible for extracting these pathogens from their systems. 

The unique interaction between checklists, humans, machines, and the operational environment, 
makes the checklist problem a true human factors issue. Nevertheless, the human factors aspects of 
this device and procedure is still ignored by many. But the price of this ignorance is much too often 
tragically established. 
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APPENDIX A 


PROPOSED GUIDELINES 


In this appendix the authors propose several guidelines for designing and using flight-deck 
checklists. These considerations are not specifications, and some when applied individually may 
conflict. Therefore, each should be carefully evaluated for its relevance to operational constraints 
and the checklist philosophy-of-use in any specific airline operation. The section in the report 
which explains the rationale for each guideline is given in parenthesis. 

(1) Every effort should be made to avoid using the checklist as a "site" for resolving discipline 
problems. (3.2.3.) 

(2) Standardization of checklists between fleets has many advantages, but this should be done 
carefully to prevent inappropriately imposing a checklist sequence and concept of one 
aircraft type on another. (3.3.) 

(3) Airlines should attempt to standardize the names assigned to controls and displays between 
different fleets. (6.2.4.) 

(4) Checklist responses should portray the desired status or the value of the item being 
considered (not just "checked" or "set"). (6.2.3.) 

(5) The use of hands and fingers to touch appropriate controls, switches, and displays while 
conducting the checklist is recommended. (7.2.2) 

(6) The completion call of a task-checklist should be written as the last item on the checklist, 
allowing all crew members to move mentally from the checklist to other activities with the 
assurance of all pilots that the task-checklist has been completed. (5.3.) 

(7) A long checklist should be subdivided to smaller task-checklists or chunks that can be 
associated with systems and functions within the cockpit. For example, a BEFORE 
START checklist can easily grow to be very lengthy. If so, it can be subdivided as 
suggested above. (7.1.1.) 

(8) Sequencing of checklist items should follow the "geographical" organization of the items in 
the cockpit, and be performed in a logical flow. Training departments should provide a 
pictorial scheme of this flow for training purposes. (7.1. and 7.2.) 

(9) Checklist items should be sequenced in parallel to internal and external activities that require 
input from out-of-cockpit agents such as cabin crew, ground crew, fuelers, and gate 
agents. (7.2., 5.4., and 8.2.2.) 

(10) The most critical items on the task-checklist should be listed as close as possible to the 
beginning of the task-checklist, in order to increase the likelihood of completing the task 
before interruptions may occur. We note that this guideline could be in conflict with Nos. 
(8) and (9) above. In most cases where this occurs, this guideline (10) should take 
precedence. (7.2., and 8.2.2.) 
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(1 1) Critical checklist items such as flaps/slats, trim, etc., that might be reset prior to takeoff due 
to new information should be duplicated between task-checklists. (7 .2.) 

(12) Checklists should be designed in such a way that they will not be tightly coupled with other 
tasks. Every effort should be made to provide buffers for recovery from failure, and a way 
to "take up the slack" if checklist completion does not keep pace with the external 
operation. (8.2.) 

(13) The TAXI checklist should be completed as close as possible to the gate and as far away as 
possible from the active runway(s) and adjacent taxiways. (8.2.) 

(14) Flight crews should be made aware that the checklist procedure is highly susceptible to 
production pressures. These pressures "set the stage" for errors by encouraging 
substandard performance, and later may lead some to relegate checklist procedures to 
second level of importance, or not use them at all in order to save time. (8.2.3.) 

(15) FAA officials, particularly Principal Operations Inspectors, should be sensitive to cultural, 
traditional, and philosophical factors in airline companies and their effect on checklists 
submitted for their approval. There should be no compromise, however, regarding the 
critical "killer" items. (3.) 

(16) Likewise, when a merger occurs, checklists of the acquired airline should be carefully 
examined for their differences. Knowledge gained by the acquired airline in operating a 
specific model should not be ignored. Differences in concepts and operating procedures 
should be resolved in a manner that enhances safe checklist behavior of all crew members. 
(4.) 
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